Create README.md
Browse filesAssociative memory with learned decision tree routing. Store memories, trees learn optimal paths based on retrieval success. Experimental research code - functional but unoptimized. Part of The Classics Revival collection.
README.md
ADDED
|
@@ -0,0 +1,98 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
tags:
|
| 3 |
+
- memory-augmented
|
| 4 |
+
- decision-trees
|
| 5 |
+
- associative-memory
|
| 6 |
+
- classics-revival
|
| 7 |
+
- experimental
|
| 8 |
+
license: apache-2.0
|
| 9 |
+
library_name: pytorch
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
# Memory Forest - The Classics Revival
|
| 13 |
+
|
| 14 |
+
**Associative Memory with Learned Routing Through Decision Trees**
|
| 15 |
+
|
| 16 |
+
**Experimental Research Code** - Functional but unoptimized, expect rough edges
|
| 17 |
+
|
| 18 |
+
## What Is This?
|
| 19 |
+
|
| 20 |
+
Memory Forest combines decision tree routing with associative hash buckets to create a scalable memory system. Instead of searching all memory linearly, learned decision trees route queries to the most relevant memory buckets.
|
| 21 |
+
|
| 22 |
+
**Core Innovation**: Trees learn to route based on retrieval success, creating adaptive memory organization that gets better over time.
|
| 23 |
+
|
| 24 |
+
## Architecture Highlights
|
| 25 |
+
|
| 26 |
+
- **Learned Routing**: Decision trees adapt based on retrieval performance
|
| 27 |
+
- **Associative Storage**: Hash buckets with similarity-based clustering
|
| 28 |
+
- **Ensemble Retrieval**: Multiple trees vote on best memories
|
| 29 |
+
- **Eviction Policies**: Automatic memory management
|
| 30 |
+
- **Scalable Design**: O(log n) routing instead of O(n) search
|
| 31 |
+
|
| 32 |
+
## Quick Start
|
| 33 |
+
```python
|
| 34 |
+
from memory_forest import MemoryForest
|
| 35 |
+
|
| 36 |
+
# Create memory system
|
| 37 |
+
forest = MemoryForest(
|
| 38 |
+
input_dim=64,
|
| 39 |
+
num_trees=3,
|
| 40 |
+
max_depth=4,
|
| 41 |
+
bucket_size=32
|
| 42 |
+
)
|
| 43 |
+
|
| 44 |
+
# Store some memories
|
| 45 |
+
features = torch.randn(100, 64)
|
| 46 |
+
forest.store(features)
|
| 47 |
+
|
| 48 |
+
# Retrieve similar items
|
| 49 |
+
query = torch.randn(5, 64)
|
| 50 |
+
results = forest.retrieve(query, top_k=3)
|
| 51 |
+
```
|
| 52 |
+
## Current Status
|
| 53 |
+
- **Working**: Core routing, storage, retrieval, adaptive learning
|
| 54 |
+
- **Rough Edges**: No benchmarking, minimal optimization, needs profiling
|
| 55 |
+
- **Still Missing**: Advanced eviction policies, memory compression, distributed storage
|
| 56 |
+
- **Performance**: Functional on small datasets
|
| 57 |
+
- **Memory Usage**: Not optimized, expect high RAM usage
|
| 58 |
+
- **Speed**: Baseline implementation, significant optimization possible
|
| 59 |
+
|
| 60 |
+
## Mathematical Foundation
|
| 61 |
+
The decision trees learn routing functions f: R^d -> {0,1,...,B-1} where B is the number of buckets. Trees update based on retrieval success using simple gradient-free optimization:
|
| 62 |
+
```python
|
| 63 |
+
success_rate = retrieved_similarity / max_possible_similarity
|
| 64 |
+
tree_update ∝ success_rate * path_taken
|
| 65 |
+
```
|
| 66 |
+
Associative buckets use learnable hash functions with Hebbian-style updates for similarity clustering.
|
| 67 |
+
|
| 68 |
+
## Research Applications
|
| 69 |
+
- **Large-scale retrieval systems**
|
| 70 |
+
- **Adaptive memory architectures**
|
| 71 |
+
- **Decision tree optimization**
|
| 72 |
+
- **Associative memory research**
|
| 73 |
+
- **Hierarchical clustering**
|
| 74 |
+
|
| 75 |
+
## Installation
|
| 76 |
+
``` python
|
| 77 |
+
pip install torch numpy
|
| 78 |
+
# Download memory_forest.py from this repo
|
| 79 |
+
```
|
| 80 |
+
## The Classics Revival Collection
|
| 81 |
+
|
| 82 |
+
Memory Forest is part of a larger exploration of foundational algorithms enhanced with modern neural techniques:
|
| 83 |
+
|
| 84 |
+
- Evolutionary Turing Machine
|
| 85 |
+
- Hebbian Bloom Filter
|
| 86 |
+
- Hopfield Decision Graph
|
| 87 |
+
- Liquid Bayes Chain
|
| 88 |
+
- Liquid State Space Model
|
| 89 |
+
- ** Memory Forest** ← You are here
|
| 90 |
+
|
| 91 |
+
## Citation
|
| 92 |
+
```bibtex
|
| 93 |
+
@misc{memoryforest2025,
|
| 94 |
+
title={Memory Forest: Associative Memory with Learned Routing},
|
| 95 |
+
author={Jae Parker 𓅸 1990two},
|
| 96 |
+
year={2025},
|
| 97 |
+
note={Part of The Classics Revival Collection}
|
| 98 |
+
}
|