Spaces:
Sleeping
Sleeping
Upload folder using huggingface_hub
Browse files
app.py
CHANGED
|
@@ -218,7 +218,7 @@ HEADER_HTML = '''
|
|
| 218 |
margin: 5px 0 0 0;
|
| 219 |
font-size: 1.1em;
|
| 220 |
font-weight: 300;
|
| 221 |
-
">
|
| 222 |
</div>
|
| 223 |
</div>
|
| 224 |
|
|
@@ -2137,24 +2137,69 @@ it learns patterns by updating weights during inference.
|
|
| 2137 |
- **Checkpoints**: Save/restore learned state via Docker volumes
|
| 2138 |
- **Container-Native**: Designed for orchestrated deployment
|
| 2139 |
|
| 2140 |
-
|
| 2141 |
|
| 2142 |
-
|
| 2143 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2144 |
|
| 2145 |
-
|
| 2146 |
-
-
|
| 2147 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2148 |
|
| 2149 |
This project demonstrates the ability to:
|
| 2150 |
1. Read cutting-edge research (Titans paper)
|
| 2151 |
2. Implement it correctly (PyTorch TTT)
|
| 2152 |
3. Productionize it (Docker, MCP, CI/CD)
|
| 2153 |
-
4.
|
| 2154 |
-
|
| 2155 |
-
**Contact:** [macayaven@gmail.com](mailto:macayaven@gmail.com)
|
| 2156 |
-
|
| 2157 |
-
**GitHub:** [macayaven/docker-neural-memory](https://github.com/macayaven/docker-neural-memory)
|
| 2158 |
"""
|
| 2159 |
|
| 2160 |
|
|
|
|
| 218 |
margin: 5px 0 0 0;
|
| 219 |
font-size: 1.1em;
|
| 220 |
font-weight: 300;
|
| 221 |
+
">Test-Time Training: Evolving LLMs from data hoarders to knowledge creators</p>
|
| 222 |
</div>
|
| 223 |
</div>
|
| 224 |
|
|
|
|
| 2137 |
- **Checkpoints**: Save/restore learned state via Docker volumes
|
| 2138 |
- **Container-Native**: Designed for orchestrated deployment
|
| 2139 |
|
| 2140 |
+
---
|
| 2141 |
|
| 2142 |
+
## Limitations
|
| 2143 |
+
|
| 2144 |
+
This is a **demonstration project**, not a production-ready system:
|
| 2145 |
+
|
| 2146 |
+
| Component | Current State | Production Would Need |
|
| 2147 |
+
|-----------|---------------|----------------------|
|
| 2148 |
+
| **RAG Implementation** | Simplified keyword matching | Vector embeddings + semantic search (FAISS, Pinecone) |
|
| 2149 |
+
| **Neural Memory** | Basic 2-layer MLP | Deeper architecture, attention mechanisms |
|
| 2150 |
+
| **Scalability** | Single-user demo | Distributed inference, GPU optimization |
|
| 2151 |
+
| **Evaluation** | Qualitative comparison | Benchmarks, ablation studies, metrics |
|
| 2152 |
+
| **Memory Capacity** | ~250K parameters | Larger models, hierarchical memory |
|
| 2153 |
+
|
| 2154 |
+
The RAG comparison uses simple word overlap scoring to demonstrate *why* keyword-based retrieval fails for pattern inference. A production RAG system would use proper embeddings and vector similarity search.
|
| 2155 |
+
|
| 2156 |
+
---
|
| 2157 |
+
|
| 2158 |
+
## Acknowledgments
|
| 2159 |
+
|
| 2160 |
+
This project builds on the work of brilliant researchers:
|
| 2161 |
|
| 2162 |
+
**Core Research:**
|
| 2163 |
+
- **Titans: Learning to Memorize at Test Time** (Google, Dec 2024) — [arXiv:2501.00663](https://arxiv.org/abs/2501.00663)
|
| 2164 |
+
- Ali Behrouz, Peilin Zhong, Vahab Mirrokni
|
| 2165 |
+
- **Learning to (Learn at Test Time): RNNs with Expressive Hidden States** (Stanford/Meta, Jul 2024) — [arXiv:2407.04620](https://arxiv.org/abs/2407.04620)
|
| 2166 |
+
- Yu Sun, Xinhao Li, Karan Dalal, et al.
|
| 2167 |
+
|
| 2168 |
+
**Frameworks & Tools:**
|
| 2169 |
+
- [PyTorch](https://pytorch.org/) — The foundation for neural memory implementation
|
| 2170 |
+
- [Gradio](https://gradio.app/) — Interactive demo interface
|
| 2171 |
+
- [HuggingFace](https://huggingface.co/) — Model hosting and inference API
|
| 2172 |
+
- [Model Context Protocol](https://modelcontextprotocol.io/) — Claude Desktop integration
|
| 2173 |
+
|
| 2174 |
+
**Inspiration:**
|
| 2175 |
+
- The broader ML community exploring alternatives to attention-based memory
|
| 2176 |
+
- Open-source contributors who make research accessible
|
| 2177 |
+
|
| 2178 |
+
---
|
| 2179 |
+
|
| 2180 |
+
## Next Steps
|
| 2181 |
+
|
| 2182 |
+
Potential improvements for future iterations:
|
| 2183 |
+
|
| 2184 |
+
1. **Real RAG Baseline**: Integrate sentence-transformers + FAISS for proper semantic retrieval comparison
|
| 2185 |
+
2. **Attention-Based Memory**: Implement the full Titans architecture with neural long-term memory gates
|
| 2186 |
+
3. **Benchmarking**: Add quantitative evaluation on standard memory tasks (bAbI, etc.)
|
| 2187 |
+
4. **Multi-Modal Support**: Extend to image/audio observations
|
| 2188 |
+
5. **Distributed Memory**: Explore memory sharing across multiple agents
|
| 2189 |
+
6. **Fine-Grained Forgetting**: Implement selective memory consolidation/pruning
|
| 2190 |
+
|
| 2191 |
+
---
|
| 2192 |
+
|
| 2193 |
+
## Built By
|
| 2194 |
+
|
| 2195 |
+
**Carlos Crespo Macaya**
|
| 2196 |
+
AI Engineer — GenAI Systems & Applied MLOps
|
| 2197 |
|
| 2198 |
This project demonstrates the ability to:
|
| 2199 |
1. Read cutting-edge research (Titans paper)
|
| 2200 |
2. Implement it correctly (PyTorch TTT)
|
| 2201 |
3. Productionize it (Docker, MCP, CI/CD)
|
| 2202 |
+
4. Communicate it effectively (this demo)
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2203 |
"""
|
| 2204 |
|
| 2205 |
|