Harry00 commited on
Commit
bf25c65
Β·
verified Β·
1 Parent(s): 29b620f

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +133 -0
README.md ADDED
@@ -0,0 +1,133 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Morpho-Logic Engine (MLE) β€” Adaptive Learning System
2
+
3
+ ## Overview
4
+
5
+ The **Morpho-Logic Engine (MLE)** is a high-dimensional sparse distributed memory system with energy-based dynamics, optimized for CPU performance through bit-slicing SIMD operations. It learns continuously during inference without classical backpropagation, using purely local, energy-driven updates.
6
+
7
+ ## Core Architecture
8
+
9
+ The system comprises five integrated modules that co-evolve during operation:
10
+
11
+ ### 1. Memory β€” Adaptive Sparse Address Table
12
+ - **4096-bit binary vectors** with target sparsity ~5% (~200 active bits)
13
+ - **Dynamic creation**: new vectors spawn for recurrent or under-represented patterns
14
+ - **Fusion & specialization**: close vectors merge; context-dependent specializations branch off
15
+ - **Local reorganization**: semantic neighborhood coherence is improved iteratively
16
+ - **Controlled forgetting**: pruning of under-used entries prevents drift
17
+
18
+ ### 2. Routing β€” Hamming Distance + Bit-Slicing SIMD
19
+ - Vectors packed into **64 Γ— uint64** slices
20
+ - **Parallel Hamming distance** computation via bit-twiddling popcount
21
+ - **Inverted index** per slice for sub-linear candidate retrieval
22
+ - **Learned route cache**: frequently traversed query→neighbor mappings are memorized
23
+
24
+ ### 3. Binding β€” Circular Convolution
25
+ - **Role-filler binding** via circular convolution in frequency domain (FFT)
26
+ - **Structure composition**: multiple role-filler pairs superposed into composite vectors
27
+ - **Robust unbinding**: recover fillers from bound representations
28
+
29
+ ### 4. Energy Landscape β€” Learnable Coherence Function
30
+ - **Hamming energy**: local coherence via neighbor distances
31
+ - **Hebbian-like associations**: co-occurring vectors in low-energy states strengthen links
32
+ - **Anti-Hebbian for instability**: high-energy configurations weaken spurious associations
33
+ - **Adaptive biases**: per-bit biases shift based on experience
34
+ - **No global gradient**: all updates are purely local
35
+
36
+ ### 5. Inference β€” Online Learning through Energy Minimization
37
+ - **Stochastic bit-flip descent** with simulated annealing temperature schedule
38
+ - **Metropolis-Hastings acceptance** for exploration/exploitation balance
39
+ - **Learning during inference**: associations, biases, and routes update at every iteration
40
+ - **Post-inference reinforcement**: stable low-energy trajectories are consolidated
41
+
42
+ ## Key Capabilities
43
+
44
+ ### Continuous Online Learning
45
+ The system learns while it reasons. Every inference pass updates:
46
+ - Vector co-activation weights
47
+ - Energy landscape associations
48
+ - Routing cache entries
49
+ - Memory structure (creation, fusion, specialization)
50
+
51
+ ### Generalization through Composition
52
+ - **Binding/unbinding** enables compositional reasoning
53
+ - **Pattern abstraction** detects recurrent low-energy trajectories and compiles them into new memory units
54
+ - **Structure reuse**: existing sub-patterns are recycled in novel contexts
55
+
56
+ ### Semantic Coherence
57
+ Local reorganization ensures vectors that are close in Hamming space correspond to semantically related concepts. Coherence score is continuously monitored.
58
+
59
+ ### CPU-Optimized Performance
60
+ - All core operations use vectorized NumPy and JIT-compiled Numba kernels
61
+ - No dense matrix multiplications
62
+ - Bit-slicing reduces memory bandwidth by 64Γ—
63
+ - Hamming distances computed via XOR + popcount
64
+
65
+ ## Benchmark Results
66
+
67
+ ```
68
+ Learning confirmed: βœ“ Energy decreased with experience
69
+ Binding accuracy: 100% (10/10)
70
+ Semantic coherence: 0.996
71
+ Avg inference time: ~540 ms
72
+ Memory growth: controlled (auto-pruning)
73
+ Convergence rate: ~78%
74
+ ```
75
+
76
+ ## Usage
77
+
78
+ ```python
79
+ from mle import MLESystem
80
+ import numpy as np
81
+
82
+ # Initialize
83
+ mle = MLESystem(
84
+ memory_capacity=2000,
85
+ online_learning=True,
86
+ temperature=0.5,
87
+ )
88
+
89
+ # Create a sparse input vector
90
+ vec = np.zeros(4096, dtype=np.uint8)
91
+ vec[np.random.choice(4096, size=200, replace=False)] = 1
92
+
93
+ # Process (inference + learning)
94
+ result = mle.process(vec)
95
+ print(f"Converged: {result.converged}")
96
+ print(f"Energy: {result.energy_trajectory[-1]:.1f}")
97
+
98
+ # Query neighbors
99
+ neighbors = mle.query(vec, k=5)
100
+
101
+ # Check system health
102
+ mle.print_summary()
103
+ ```
104
+
105
+ ## Directory Structure
106
+
107
+ ```
108
+ mle/
109
+ β”œβ”€β”€ __init__.py # Package exports
110
+ β”œβ”€β”€ memory.py # Adaptive Sparse Address Table
111
+ β”œβ”€β”€ routing.py # Hamming router with bit-slicing
112
+ β”œβ”€β”€ binding.py # Circular convolution binder
113
+ β”œβ”€β”€ energy.py # Learnable energy landscape
114
+ β”œβ”€β”€ inference.py # Online learning inference engine
115
+ β”œβ”€β”€ mle_system.py # Full system integration + metrics
116
+ └── tests.py # Comprehensive benchmark suite
117
+ ```
118
+
119
+ ## Design Principles
120
+
121
+ 1. **Locality**: every update touches only a neighborhood, no global passes
122
+ 2. **Sparsity**: 5% active bits β†’ 95% of computation skipped implicitly
123
+ 3. **Energy as teacher**: low energy = good, high energy = bad, no labels needed
124
+ 4. **Memory is computation**: the memory table *is* the model; no separate weights
125
+ 5. **Continuity**: training and inference are the same operation
126
+
127
+ ## Future Directions
128
+
129
+ - Multi-resolution binding for hierarchical structures
130
+ - Cross-modal binding (vision + language in shared space)
131
+ - Energy landscape visualization and analysis
132
+ - Distributed memory shards for web-scale operation
133
+ - Integration with LLM token embeddings for hybrid reasoning