We will show how an algorithmic model of intelligence can be expressed concisely in Julia, closely mapping the definitions in the literature that describes it. This new package that implements a biologically constrained model of mammal intelligence can serve as a theoretical framework to guide neuroscience research or perform machine learning tasks like time series prediction and anomaly detection.
We describe the design of HierarchicalTemporalMemory.jl, a Julia package that implements an algorithmic model of intelligence, the Hierarchical Temporal Memory. The main algorithms of this model, the Spatial Pooler and Temporal (Sequence) Memory, are described with concise tensor expressions in:
- The HTM Spatial Pooler—A Neocortical Algorithm for Online Sparse Distributed Coding
- Continuous Online Sequence Learning with an Unsupervised Neural Network Model
The official implementation of this model in Python (NUPIC) serves as a reference point and as source of truth for many implementation details, but it doesn't take advantage of the data-driven design that the source material encourages and ends up quite verbose. This implementation uses Julia's expressivity to remain faithful to the papers' terminology, attempting to express the algorithms more simply and concisely, and thus instigate further research on them. Some essential features of this expressivity are broadcasting, duck-typed interfaces and unicode source code.
The Hierarchical Temporal Memory (HTM) is a biologically constrained theory, aiming primarily to model the function of the neocortex (a structure of the human brain), and as a secondary goal, machine learning applications. The algorithms expose many fundamental properties which we use to test this implementation.
In contrast to many established neural network models, the HTM neuron performs coincidence detection across multiple input "dendrites". HTM is actually closer to a spiking neural network, with binary synapses and signals, that encodes information in population codes. Spatial pooling and the temporal memory are both unsupervised processes that adapt continuously; together, they learn to identify sequences in noisy input time series.
HTM theory is not yet complete, lacking a definitive way to stabilize sequence representations and compose small models. Exactly for this reason, we believe that a concise and high level model can accelerate the research.