msrcam/claudia-memory-pipeline

Claudia Memory Pipeline

Weight-only memory for large language models. No RAG, no system prompts, no external databases. Facts learned in conversation are permanently stored in model weights via LoRA.

Key Results

Metric	Score
Single-session recall (best)	14/15 (93%)
Fresh-model recall	12/15 (80%)
Cross-session persistence	9/11 (82%)
Self-quiz impact	21% → 74% recall

How It Works

1. Self-Quiz Pair Generation — Model generates Q&A pairs about facts it just learned, plus contrastive pairs to prevent entity confusion
2. Two-Phase LoRA Absorption — Phase 1 trains positive facts (entity-clustered), Phase 2 targets only confused entities with contrastive data
3. Phase 3 Stubborn Retry — Persistently confused entities get focused reinforcement (3x correct + contrastive, max 3 rounds)
4. Cascade Distillation — Teacher logits cached at session end, distilled into student at next session start for long-term retention

Model

All experiments on Qwen3-Omni-30B-A3B (abliterated, MoE: 128 experts, top-8, 48 thinker layers).

Files

• persistent_absorber.py — Production pipeline (~1800 lines). Load model, chat, learn, save, resume.
• MEMORY_PIPELINE_PAPER.md — Full research paper with all parameters, experiments, findings, and reproduction guide.
• SESSION4_REPORT.md — Detailed session 4 experiment series (4a through 4f).
• HANDOFF.md — Deployment guide for Vast.ai instances.
• setup.sh — Instance setup script.
• tests/ — All test scripts from sessions 4-5.

LoRA Config

rank=128, alpha=256, targets=q_proj/k_proj/v_proj/o_proj
LR: attention=6e-5, FFN=3e-4
10 steps per exchange, batch_size=1

Quick Start

See HANDOFF.md for full deployment instructions. Requires A100 80GB (64GB minimum for thinker-only).

# On Vast.ai A100 instance
git clone https://huggingface.co/msrcam/claudia-memory-pipeline
cd claudia-memory-pipeline
bash setup.sh
python persistent_absorber.py

Author

Matt (msrcam) — March 2026