macayaven commited on
Commit
c42b81e
·
verified ·
1 Parent(s): 6d93c79

Upload folder using huggingface_hub

Browse files
Files changed (1) hide show
  1. app.py +57 -12
app.py CHANGED
@@ -218,7 +218,7 @@ HEADER_HTML = '''
218
  margin: 5px 0 0 0;
219
  font-size: 1.1em;
220
  font-weight: 300;
221
- ">Real Test-Time Training Not a Simulation</p>
222
  </div>
223
  </div>
224
 
@@ -2137,24 +2137,69 @@ it learns patterns by updating weights during inference.
2137
  - **Checkpoints**: Save/restore learned state via Docker volumes
2138
  - **Container-Native**: Designed for orchestrated deployment
2139
 
2140
- ### Built By
2141
 
2142
- **Carlos Crespo Macaya**
2143
- AI Engineer - GenAI Systems & Applied MLOps
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2144
 
2145
- - 10+ years production ML experience
2146
- - Expert in Docker, Kubernetes, MCP servers
2147
- - Currently at HP AICoE building multi-agent systems
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2148
 
2149
  This project demonstrates the ability to:
2150
  1. Read cutting-edge research (Titans paper)
2151
  2. Implement it correctly (PyTorch TTT)
2152
  3. Productionize it (Docker, MCP, CI/CD)
2153
- 4. Make it compelling (this demo)
2154
-
2155
- **Contact:** [macayaven@gmail.com](mailto:macayaven@gmail.com)
2156
-
2157
- **GitHub:** [macayaven/docker-neural-memory](https://github.com/macayaven/docker-neural-memory)
2158
  """
2159
 
2160
 
 
218
  margin: 5px 0 0 0;
219
  font-size: 1.1em;
220
  font-weight: 300;
221
+ ">Test-Time Training: Evolving LLMs from data hoarders to knowledge creators</p>
222
  </div>
223
  </div>
224
 
 
2137
  - **Checkpoints**: Save/restore learned state via Docker volumes
2138
  - **Container-Native**: Designed for orchestrated deployment
2139
 
2140
+ ---
2141
 
2142
+ ## Limitations
2143
+
2144
+ This is a **demonstration project**, not a production-ready system:
2145
+
2146
+ | Component | Current State | Production Would Need |
2147
+ |-----------|---------------|----------------------|
2148
+ | **RAG Implementation** | Simplified keyword matching | Vector embeddings + semantic search (FAISS, Pinecone) |
2149
+ | **Neural Memory** | Basic 2-layer MLP | Deeper architecture, attention mechanisms |
2150
+ | **Scalability** | Single-user demo | Distributed inference, GPU optimization |
2151
+ | **Evaluation** | Qualitative comparison | Benchmarks, ablation studies, metrics |
2152
+ | **Memory Capacity** | ~250K parameters | Larger models, hierarchical memory |
2153
+
2154
+ The RAG comparison uses simple word overlap scoring to demonstrate *why* keyword-based retrieval fails for pattern inference. A production RAG system would use proper embeddings and vector similarity search.
2155
+
2156
+ ---
2157
+
2158
+ ## Acknowledgments
2159
+
2160
+ This project builds on the work of brilliant researchers:
2161
 
2162
+ **Core Research:**
2163
+ - **Titans: Learning to Memorize at Test Time** (Google, Dec 2024) — [arXiv:2501.00663](https://arxiv.org/abs/2501.00663)
2164
+ - Ali Behrouz, Peilin Zhong, Vahab Mirrokni
2165
+ - **Learning to (Learn at Test Time): RNNs with Expressive Hidden States** (Stanford/Meta, Jul 2024) — [arXiv:2407.04620](https://arxiv.org/abs/2407.04620)
2166
+ - Yu Sun, Xinhao Li, Karan Dalal, et al.
2167
+
2168
+ **Frameworks & Tools:**
2169
+ - [PyTorch](https://pytorch.org/) — The foundation for neural memory implementation
2170
+ - [Gradio](https://gradio.app/) — Interactive demo interface
2171
+ - [HuggingFace](https://huggingface.co/) — Model hosting and inference API
2172
+ - [Model Context Protocol](https://modelcontextprotocol.io/) — Claude Desktop integration
2173
+
2174
+ **Inspiration:**
2175
+ - The broader ML community exploring alternatives to attention-based memory
2176
+ - Open-source contributors who make research accessible
2177
+
2178
+ ---
2179
+
2180
+ ## Next Steps
2181
+
2182
+ Potential improvements for future iterations:
2183
+
2184
+ 1. **Real RAG Baseline**: Integrate sentence-transformers + FAISS for proper semantic retrieval comparison
2185
+ 2. **Attention-Based Memory**: Implement the full Titans architecture with neural long-term memory gates
2186
+ 3. **Benchmarking**: Add quantitative evaluation on standard memory tasks (bAbI, etc.)
2187
+ 4. **Multi-Modal Support**: Extend to image/audio observations
2188
+ 5. **Distributed Memory**: Explore memory sharing across multiple agents
2189
+ 6. **Fine-Grained Forgetting**: Implement selective memory consolidation/pruning
2190
+
2191
+ ---
2192
+
2193
+ ## Built By
2194
+
2195
+ **Carlos Crespo Macaya**
2196
+ AI Engineer — GenAI Systems & Applied MLOps
2197
 
2198
  This project demonstrates the ability to:
2199
  1. Read cutting-edge research (Titans paper)
2200
  2. Implement it correctly (PyTorch TTT)
2201
  3. Productionize it (Docker, MCP, CI/CD)
2202
+ 4. Communicate it effectively (this demo)
 
 
 
 
2203
  """
2204
 
2205