Upload README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,133 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Morpho-Logic Engine (MLE) β Adaptive Learning System
|
| 2 |
+
|
| 3 |
+
## Overview
|
| 4 |
+
|
| 5 |
+
The **Morpho-Logic Engine (MLE)** is a high-dimensional sparse distributed memory system with energy-based dynamics, optimized for CPU performance through bit-slicing SIMD operations. It learns continuously during inference without classical backpropagation, using purely local, energy-driven updates.
|
| 6 |
+
|
| 7 |
+
## Core Architecture
|
| 8 |
+
|
| 9 |
+
The system comprises five integrated modules that co-evolve during operation:
|
| 10 |
+
|
| 11 |
+
### 1. Memory β Adaptive Sparse Address Table
|
| 12 |
+
- **4096-bit binary vectors** with target sparsity ~5% (~200 active bits)
|
| 13 |
+
- **Dynamic creation**: new vectors spawn for recurrent or under-represented patterns
|
| 14 |
+
- **Fusion & specialization**: close vectors merge; context-dependent specializations branch off
|
| 15 |
+
- **Local reorganization**: semantic neighborhood coherence is improved iteratively
|
| 16 |
+
- **Controlled forgetting**: pruning of under-used entries prevents drift
|
| 17 |
+
|
| 18 |
+
### 2. Routing β Hamming Distance + Bit-Slicing SIMD
|
| 19 |
+
- Vectors packed into **64 Γ uint64** slices
|
| 20 |
+
- **Parallel Hamming distance** computation via bit-twiddling popcount
|
| 21 |
+
- **Inverted index** per slice for sub-linear candidate retrieval
|
| 22 |
+
- **Learned route cache**: frequently traversed queryβneighbor mappings are memorized
|
| 23 |
+
|
| 24 |
+
### 3. Binding β Circular Convolution
|
| 25 |
+
- **Role-filler binding** via circular convolution in frequency domain (FFT)
|
| 26 |
+
- **Structure composition**: multiple role-filler pairs superposed into composite vectors
|
| 27 |
+
- **Robust unbinding**: recover fillers from bound representations
|
| 28 |
+
|
| 29 |
+
### 4. Energy Landscape β Learnable Coherence Function
|
| 30 |
+
- **Hamming energy**: local coherence via neighbor distances
|
| 31 |
+
- **Hebbian-like associations**: co-occurring vectors in low-energy states strengthen links
|
| 32 |
+
- **Anti-Hebbian for instability**: high-energy configurations weaken spurious associations
|
| 33 |
+
- **Adaptive biases**: per-bit biases shift based on experience
|
| 34 |
+
- **No global gradient**: all updates are purely local
|
| 35 |
+
|
| 36 |
+
### 5. Inference β Online Learning through Energy Minimization
|
| 37 |
+
- **Stochastic bit-flip descent** with simulated annealing temperature schedule
|
| 38 |
+
- **Metropolis-Hastings acceptance** for exploration/exploitation balance
|
| 39 |
+
- **Learning during inference**: associations, biases, and routes update at every iteration
|
| 40 |
+
- **Post-inference reinforcement**: stable low-energy trajectories are consolidated
|
| 41 |
+
|
| 42 |
+
## Key Capabilities
|
| 43 |
+
|
| 44 |
+
### Continuous Online Learning
|
| 45 |
+
The system learns while it reasons. Every inference pass updates:
|
| 46 |
+
- Vector co-activation weights
|
| 47 |
+
- Energy landscape associations
|
| 48 |
+
- Routing cache entries
|
| 49 |
+
- Memory structure (creation, fusion, specialization)
|
| 50 |
+
|
| 51 |
+
### Generalization through Composition
|
| 52 |
+
- **Binding/unbinding** enables compositional reasoning
|
| 53 |
+
- **Pattern abstraction** detects recurrent low-energy trajectories and compiles them into new memory units
|
| 54 |
+
- **Structure reuse**: existing sub-patterns are recycled in novel contexts
|
| 55 |
+
|
| 56 |
+
### Semantic Coherence
|
| 57 |
+
Local reorganization ensures vectors that are close in Hamming space correspond to semantically related concepts. Coherence score is continuously monitored.
|
| 58 |
+
|
| 59 |
+
### CPU-Optimized Performance
|
| 60 |
+
- All core operations use vectorized NumPy and JIT-compiled Numba kernels
|
| 61 |
+
- No dense matrix multiplications
|
| 62 |
+
- Bit-slicing reduces memory bandwidth by 64Γ
|
| 63 |
+
- Hamming distances computed via XOR + popcount
|
| 64 |
+
|
| 65 |
+
## Benchmark Results
|
| 66 |
+
|
| 67 |
+
```
|
| 68 |
+
Learning confirmed: β Energy decreased with experience
|
| 69 |
+
Binding accuracy: 100% (10/10)
|
| 70 |
+
Semantic coherence: 0.996
|
| 71 |
+
Avg inference time: ~540 ms
|
| 72 |
+
Memory growth: controlled (auto-pruning)
|
| 73 |
+
Convergence rate: ~78%
|
| 74 |
+
```
|
| 75 |
+
|
| 76 |
+
## Usage
|
| 77 |
+
|
| 78 |
+
```python
|
| 79 |
+
from mle import MLESystem
|
| 80 |
+
import numpy as np
|
| 81 |
+
|
| 82 |
+
# Initialize
|
| 83 |
+
mle = MLESystem(
|
| 84 |
+
memory_capacity=2000,
|
| 85 |
+
online_learning=True,
|
| 86 |
+
temperature=0.5,
|
| 87 |
+
)
|
| 88 |
+
|
| 89 |
+
# Create a sparse input vector
|
| 90 |
+
vec = np.zeros(4096, dtype=np.uint8)
|
| 91 |
+
vec[np.random.choice(4096, size=200, replace=False)] = 1
|
| 92 |
+
|
| 93 |
+
# Process (inference + learning)
|
| 94 |
+
result = mle.process(vec)
|
| 95 |
+
print(f"Converged: {result.converged}")
|
| 96 |
+
print(f"Energy: {result.energy_trajectory[-1]:.1f}")
|
| 97 |
+
|
| 98 |
+
# Query neighbors
|
| 99 |
+
neighbors = mle.query(vec, k=5)
|
| 100 |
+
|
| 101 |
+
# Check system health
|
| 102 |
+
mle.print_summary()
|
| 103 |
+
```
|
| 104 |
+
|
| 105 |
+
## Directory Structure
|
| 106 |
+
|
| 107 |
+
```
|
| 108 |
+
mle/
|
| 109 |
+
βββ __init__.py # Package exports
|
| 110 |
+
βββ memory.py # Adaptive Sparse Address Table
|
| 111 |
+
βββ routing.py # Hamming router with bit-slicing
|
| 112 |
+
βββ binding.py # Circular convolution binder
|
| 113 |
+
βββ energy.py # Learnable energy landscape
|
| 114 |
+
βββ inference.py # Online learning inference engine
|
| 115 |
+
βββ mle_system.py # Full system integration + metrics
|
| 116 |
+
βββ tests.py # Comprehensive benchmark suite
|
| 117 |
+
```
|
| 118 |
+
|
| 119 |
+
## Design Principles
|
| 120 |
+
|
| 121 |
+
1. **Locality**: every update touches only a neighborhood, no global passes
|
| 122 |
+
2. **Sparsity**: 5% active bits β 95% of computation skipped implicitly
|
| 123 |
+
3. **Energy as teacher**: low energy = good, high energy = bad, no labels needed
|
| 124 |
+
4. **Memory is computation**: the memory table *is* the model; no separate weights
|
| 125 |
+
5. **Continuity**: training and inference are the same operation
|
| 126 |
+
|
| 127 |
+
## Future Directions
|
| 128 |
+
|
| 129 |
+
- Multi-resolution binding for hierarchical structures
|
| 130 |
+
- Cross-modal binding (vision + language in shared space)
|
| 131 |
+
- Energy landscape visualization and analysis
|
| 132 |
+
- Distributed memory shards for web-scale operation
|
| 133 |
+
- Integration with LLM token embeddings for hybrid reasoning
|