APP_HANDOFF.md

# App-level handoff — composite operation of the SP model as a local assistant

This is the **operational** handoff: how the bounded-memory soft-prompt (SP) reasoner is wired into a
usable on-device app, what was built, what's verified, and where the edges are. For the *research*
handoff (training, deepKL, architecture) see [`HANDOFF.md`](HANDOFF.md); for the runbook see
[`OPERATING.md`](OPERATING.md); for the portfolio writeup [`README.md`](README.md).

> One line: a 1.5B reasoning distill runs with **O(1) bounded KV** (distant context → 32 SP vectors);
> a **tiered retrieval + memory** layer (same-session → prior-session → web) feeds verbatim facts into
> the raw window so the lossy SP is bypassed, with a **groundedness guard** that resamples contaminated
> answers. Runs at ~36 tok/s on an 8 GB Mac (MLX 4-bit).

---

## 0. FINAL SPEC (current behaviour, after all iterations)

What `tiered_rag_mlx.ChatSession.turn()` does for each user message, end to end:

**A. Classify the turn** (`intent_of`) — ONE **learned 6-class router**: a LogisticRegression on a **pure
BGE-small embedding** (`evals/intent_clf.joblib`, ~95% CV) routes each turn to exactly one action:
- **recall** — a question about the USER → personal memory only (L1/L2), never web.
- **lookup** — a general/WORLD question → web (so "who is the emperor of Japan" goes to web, not personal L1).
- **math** — a self-contained problem → solve.
- **command** — "recompute the total" → recall-log + compute.
- **fact** — a stated value/attribute → log.
- **chitchat** — greeting / conversational remark / open-ended request → respond.

This *replaces all the regex gates* (`_is_question`, `_is_factlike`, `_looks_mathy`, `compute_ref`) — one
embedding model does all routing. Training = hand-labelled core + **open-model-generated** examples
(Qwen2.5-3B-Instruct-4bit, `evals/intent_gen.py`). Regex gates are only a low-confidence/no-model fallback.
(History: regex → BGE+syntactic hybrid for small data → pure BGE once data grew → split `question`→`recall`/
`lookup` after a realistic session showed world questions hitting personal memory and hallucinating. The
**recall/lookup split was the key fix** for that — a personal vs world question is a routing distinction,
not a retrieval-threshold one. `intent_features.py` is deprecated.)

**B. Write to memory** (so future turns can recall it):
- explicit save ("remember my … is X") → **persist to disk (L2)** + reply instantly "Got it — saved." (no generation).
- a **fact-like statement** (asserts a value/attribute: "8 guests", "paint it blue") → **auto-log to L1** (same-session).
- chit-chat / knowledge-requests / questions → **NOT logged** (logging them poisons recall).
- **specific info stated INSIDE a non-recall turn** (e.g. "$120/day" inside a *math* question) → **pinned to
  working memory** by the independent specificity head (see B′), even though the turn isn't a logged fact.

**B′. Specificity head (independent, pin working memory)** — closes the gap where a specific VALUE is stated
inside a question/math turn ("if I spend **$120** a day…") and is therefore never logged, so a later
"recompute" / "what was the rate" misses it. The head is **one more linear probe on the SAME BGE vector**
already computed for routing/retrieval (`evals/specificity_clf.joblib`, 5-fold CV **AUC 0.996**), chosen via
a bake-off (`evals/specificity_probe.py`) where it beat tokenizer-fragmentation/tpc (AUC 0.87) and a
form-frequency surprisal proxy (0.30 — dead). `specific_spans(text)` enumerates cheap candidates (money,
codes, times, measures, proper-noun runs) and keeps those the probe scores ≥0.6. In `turn()`: for any intent
**except recall/lookup** (there the specific span is what's being *asked about*, e.g. "who is Naruhito" — not
user info), a turn with specific spans is **pinned** (`TieredMemory.pins`, rolling cap 12). Pins feed:
**command/compute** as primary context (the recompute needs the value), and **recall/lookup** as a
**strict (sim≥0.5) fallback only** — genuine logged facts win, so a pinned "$120 a day for 3 days" can't
derail "how long am I staying". Verified `evals/pin_test.py` (detector 9/9; pin→recall flow pass) and the
realistic session: `recompute the food total` now = **480** (was un-recoverable before, the $120 lived only
in a math turn). Train/extend: `python3.12 evals/specificity_train.py` (+ optional `specificity_gen.jsonl`).
**Follow-up `math` turns also get WM pins injected** (excluding the current turn's own pin) so a multi-turn
question that references earlier context ("she pays with $50, how much change" needs turn-1's $24, which
scrolled out of the raw window) can recover it — this was a real bug (the designed RAG didn't fire on `math`
follow-ups, only on `command`).
**The recall/lookup pin-fallback intercepts only pins NOT already logged in the session** (`p not in session`)
— a LOGGED fact ("going to Kyoto") must not shadow a genuine world question ("emperor of *Japan*", which is
geographically near "Kyoto"); logged facts are recall-reachable anyway. **POLICY COUPLING (important):** this
"working-memory-only" filter assumes the app logs ONLY facts to the session — i.e. non-fact turns are called
with `store="none"` (cf. `app_session.py` / `app_eval2.py`). If a math/chitchat turn were logged to the
session, its pinned value would stop being WM-only and the lookup→pins safety net could not fire for it.
Evaluated end-to-end in `evals/app_eval2.py` (PHASE 1 architecture scorecard **15/15**: routing + pin +
retrieval; PHASE 2 real generation). PHASE 1 caught the Kyoto/Japan over-interception bug above.

**C. Retrieve context** (only for questions / referential imperatives) — `_match` = **BGE-small semantic
cosine** (`runtime/rag.py`, 33M, CPU), lexical-overlap fallback if BGE unavailable. Handles paraphrase
("which spot did I leave the **car**?" ≈ "**parked** in bay 12", "how many **people**?" ≈ "**guests**"):
1. **L1 same-session** — semantic top-K (cap 4) of the logged facts, kept within 0.12 cosine of the best
   (a strong single match drops distractors), **ties broken by recency** so a correction wins.
2. **L2 prior-session** — same semantic match over the on-disk profile (survives a fresh process).
3. **self-contained math** (has digits) → solve it, **don't retrieve**.
4. **referential imperative** ("recompute the total") → inject the recent fact log + compute.
5. else → **L3 web** (DuckDuckGo → Wikipedia, keyless).

**D′. Recall = clean-context quote** — for a recall turn that retrieved a fact chunk, the answer is generated
by `ChatSession._clean_quote()` from an **ISOLATED context (the retrieved chunk + question only, temp 0.2, no
soft prompt, no conversation history)**, with the groundedness re-roll. Why: in a long multi-turn chat the
distractor turns' CoTs compressed into the SP can hijack a recall into computing/confabulating (e.g. quoting a
distractor's number); an isolated context removes that bias. Validated: deep recall after 6 distractors went
from a stochastic 0/3–3/3 to a reliable **3/3 across repeats** (`evals/recall_distract.py`). The retrieval
(pins/L1) was already firing; this fixed the recall *generation discipline*. (Raw SP alone confabulates 0/3,
and even full-KV base fails to self-recall — so the external retrieval injection is essential, not a crutch.)

**D. Inject + instruct** — matched facts go verbatim into the raw window:
- recall → *"quote the value verbatim; if it was corrected, use the most recent."*
- compute → *"use the fact(s) and compute."* (and the groundedness check is skipped for math).

**E. Generate** — **free generation** (no think-budget), **temp 0.6**, **rw 1024**; bounded only by a
**240 s wall-clock timeout + 2000-token per-turn cap + a 6-identical-token loop guard**.

**F. Make a real chat answer (2-pass)** — think freely; if the post-`</think>` answer is **weak**
(empty / a "…:" preamble / just a `\boxed{}`), inject **"Final answer:"** and continue, so a clean
answer always surfaces (e.g. "4000 square meters.", not an empty reply).

**G. Quality gate** — if the answer is empty/timed-out, or (for a recall turn) **not grounded** in the
retrieved context, **delete the CoT and re-roll** (`retries=2`; cheap — the hypernet recomputes the SP).

**Memory tiers:** L1 same-session (in-RAM facts) → L2 prior-session (disk jsonl) → L3 web. **Verified:**
app_demo recall 3/3, routing 5/5, web+math hit; 5- and 10-rally consistency (corrections persist; the
oldest fact fades after ~many turns). **Knobs:** `temp 0.6`, `rw 1024`, `retries 2`, `TIMEOUT_S 240`,
`MAX_NEW 2000`. **Limits:** bounded-recency (turn-1 facts fade in a long chat — the specificity head pins
*specific* values but a rolling cap-12 buffer still ages out; broad summary-memory not built), and the 1.5B
base ceiling (mis-reads/mis-computes hard problems).

---

## 0.5 Performance vs the full-KV base (GSM8K, 2026-06-10)

Does bounded SP-evict compression cost accuracy vs keeping the whole CoT in KV? Measured base (rw=8000,
full-KV) vs SP-evict (rw=1024, the shipping window) on GSM8K, interleaved, identical decoding (`evals/
gsm8k_eval.py`, `evals/gsm8k_multiturn.py`). Yardstick = the base model's OWN accuracy in-harness (NOT 100%).
- **Single-turn (n=12):** base 6/12 vs SP 8/12; **base-only losses = 0** (SP never lost a problem to base).
  SP CoT is shorter (med 1378 vs 1894 tok) and hits the 2000-tok cap less (2 vs 6) → compression keeps
  generation focused. (Caveat: short problems fit the 1024 window → SP barely compresses → not a hard test.)
- **Multi-turn (n=10, turn-2 depends on turn-1 which scrolled out of the window → the real compression test):**
  base 6/10 vs SP 6/10 (**TIED**) once the math-turn pin injection is in. **9/10 failures are `None` =
  TERMINATION** (240s timeout / 2000-tok cap hit mid-CoT) — and they hit BASE (4×) as much as SP (3×). base
  has full-KV verbatim context and fails the SAME way → the bottleneck is generation TERMINATION, NOT
  compression. Captured `None` cases re-run correctly → the model can solve them; it just often doesn't stop
  in budget (R1 CoT is long).
- **Verdict:** on the fair in-harness comparison there is **no clear accuracy loss from compression** — SP ≈
  base; what's left is the 1.5B's own ceiling + a termination artifact shared by base. (Low absolute % is
  the cap + 4-bit, not an SP deficiency.)
- **Termination fix** (`_gen_once`, 2026-06-10): on a mid-CoT timeout/cap with no usable answer, close
  `</think>` and force "Final answer:" on a small own budget (64 tok / 45 s) to salvage an answer from the
  partial CoT instead of returning None. Helps base & SP equally.

## 1. The composite pipeline (per user turn)

```
 user turn
    │
    ▼
 classify(intent)  ──► chit-chat ───────────────► answer (no retrieval)
    │                  fact ──────────────────────► save (instant "Got it — saved."), persist?→disk
    ▼ question
 TieredMemory.retrieve  (first tier over the bar wins)
    ├─ L1 same-session   (in-memory turns)
    ├─ L2 prior-session  (disk jsonl — survives a fresh process)
    ├─ self-contained math? → solve, DON'T web-search
    └─ L3 web            (DuckDuckGo → Wikipedia, urllib+bs4, no headless browser)
    │
    ▼ inject matched text verbatim into the raw window ("answer from Context only")
 bounded SP-evict generation  (force-<think>, FREE gen; 2000-tok cap + 240s timeout + loop guard — see §6.5)
    │
    ▼ answer empty / timed-out / (grounded) contaminated?  ── yes ──► delete CoT, re-roll (retries=2)
    │                                            (gen/kept hold the convo; SP recomputed each step)
    ▼ no
 answer  (only post-</think> shown; <think> scratchpad hidden)
```

## 2. App-level file map

| file | role |
|---|---|
| `tiered_rag_mlx.py` | **operational core** — `TieredMemory` (L1/L2/L3 cascade + **`pins` working memory**, fuzzy/semantic match), `ChatSession` (bounded SP-evict gen + injection + groundedness retry), learned router (`intent_of`), **specificity head (`specific_spans`, pin gate)**, `is_grounded`. Run directly → L1/L2/L3 composite test. |
| `evals/specificity_train.py` · `evals/specificity_clf.joblib` | **independent specificity head** — train + the ~20 KB BGE linear probe that detects pin-worthy specific/proper info (CV AUC 0.996). |
| `evals/specificity_probe.py` | the **bake-off** that selected BGE-probe over tpc-fragmentation / surprisal-proxy for specificity detection. |
| `evals/pin_test.py` | operational test for the specificity head + pinning (detector unit 9/9; value-in-question → later-recall flow). |
| `app_demo_mlx.py` | **two-session application simulation** + readiness report (onboarding→disk, then a fresh "today" session over a mixed 8-turn conversation). |
| `chat_mlx2.py` | recommended **interactive chat** (recipe: temp 0.6, no system prompt, force-`<think>`, free generation + empty-answer re-roll, large `rw`). |
| `mem_rag_mlx.py` | conversation-memory RAG A/B (recall a fact the SP compressed away; 0/2 → 2/2). |
| `rag_web_mlx.py` | web-grounded answer (DuckDuckGo → Wikipedia) windowed-RAG. |
| `runtime/web_search.py` | keyless search backends — `DuckDuckGoSearch` (urllib+bs4, **recommended**), `WikipediaSearch`, `PlaywrightBingSearch` (brittle, optional), `Tavily`/`Brave` (API). |
| `pooler_mlx.py` · `gen_mlx.py` · `sp_mlx.py` · `mt_mlx.py` · `build_fft_hf.py` | MLX port of the pooler + SP-evict rollout, single-turn / battery / multi-turn runners, FFT→HF builder. |

## 3. Run it

```bash
pip install mlx mlx-lm transformers torch huggingface_hub beautifulsoup4 lxml   # no Playwright needed
python build_fft_hf.py && python -m mlx_lm convert --hf-path ./fft_hf --mlx-path ./fft_mlx4 -q --q-bits 4 --q-group-size 64

python app_demo_mlx.py            # full two-session app simulation + readiness report
python tiered_rag_mlx.py         # L1/L2/L3 tier-routing composite test
python chat_mlx2.py 1000         # interactive chat
```

Minimal integration (the whole app loop):

```python
import tiered_rag_mlx as T
mem  = T.TieredMemory("user_profile.jsonl")
chat = T.ChatSession(mem, rw=512)
def on_message(text):
    if T.classify(text) == "fact":                                    # store + instant ack, no generation
        return chat.turn(text, store="persist" if T.wants_persist(text) else "session", ack_only=True)[0]
    return chat.turn(text, store="none")[0]                           # free gen; turn() retries empty/contaminated
```

## 4. What's verified (composite tests)

| test | result |
|---|---|
| **Tier routing** (`app_demo`, `tiered_rag`) | **5/5 deterministic** — chit-chat / fact-save / math / L1 / L2 / L3 all dispatch correctly, every run |
| **MEMORY recall hit-rate** (L1/L2, `app_demo`) — *the architecture's metric* | **reliable** — the SP/memory's job (hold & surface a fact across turns) lands consistently |
| web (L3) / math — *orthogonal, reported for reference* | stochastic: web = retrieval quality + base extraction; **math = pure base-1.5B arithmetic with the query fully visible (no compression) — NOT a memory test, out of scope** |

> **Scoring note (don't repeat my mistake):** the old "answer correctness X/5" lumped three unrelated
> things. What this system is measured by is **MEMORY recall** (L1/L2). A math miss is the base model's
> arithmetic, not an architecture failure — the math query is fully visible, nothing is compressed/recalled.
> Also: "HIT" = gold substring present (a *hit-rate*, a proxy), not a verified correctness rate; and any
> single `app_demo` run is 1-sample/turn (2–4/5 swing) — use N-run averages for real numbers.
| same-session recall (L1) | ✅ "B3, spot 47" |
| prior-session recall (L2, from disk, fresh process) | ✅ "POL-55821", "Aki" |
| web grounding (L3) | ✅ "Sam Altman" (after contamination retry); Mt Fuji "3,776.24 m" |
| conversation-memory RAG (`mem_rag`) | ✅ **0/2 → 2/2** (recover SP-compressed facts) |
| reasoning correctness (5 Dolphin-R1 word problems) | **SP-evict ≈ 3/5 vs base/full-KV ≈ 4/5** — same ballpark; the ceiling is the **base 1.5B**, not compression (§6.5 #7). Chat-answer fix surfaces clean answers ("4000 square meters.", "2 loads.") |
| typical miss | base-model mis-read / mis-compute, or verbose / weak instruction-following (not a memory or compression failure) |

## 5. Engineering findings & fixes (the bugs the composite tests caught)

1. **Headless Bing is brittle** → it serves a JS shell / dictionary bot-detection page. Switched the
   default web backend to **DuckDuckGo via plain `urllib`+`bs4`** (server-rendered results, no Chromium).
2. **Non-questions web-searched & contaminated** ("my room is 1408" → "1408 Film Locations" → "140813")
   → **question-gate**: only information-seeking turns retrieve.
3. **Self-contained math got web-searched** (would contaminate the calc) → **math-gate**: solve, don't search.
4. **`park` ≠ `parked`, 1-word questions missed** → **fuzzy prefix overlap** + trusted-local threshold.
5. **Fact-saves produced rambly "(no answer)"** → **instant ack** (store without generating).
6. **Recency bleed** ("Aki" answered for the CEO) → inject *"answer from the Context block only, ignore
   earlier conversation"* + a **groundedness guard** (`is_grounded`) that resamples when the answer's
   salient *novel* token isn't in the retrieved context (catches out-of-context / empty / question-echo).

## 6.5 Generation control — the long investigation (read before touching `_gen_once`)

How to bound/terminate generation was the hardest sub-problem. Final state and the reasoning, so the
next person doesn't re-run the same dead ends:

**Current committed behaviour (`tiered_rag_mlx.ChatSession`):**
- **Free generation** — no in-loop "think-budget" (we do *not* force `</think>` after N tokens).
- **2-pass chat answer (`_gen_once`)** — pass 0 thinks freely; if the post-`</think>` answer is **weak**
  (empty / ends in ':' / just a `\boxed{}`), pass 1 injects **"Final answer: "** and continues, forcing a
  clean chat answer. This fixed the dominant chat failure: the R1-distill routinely traps its answer
  *inside* `<think>` or emits only a preamble, so the user saw an empty reply even when the answer was
  computed. After the fix, math word-problems surface clean answers ("4000 square meters.", "2 loads.",
  "72 female guests…") instead of empties. `_extract_answer` is the fallback (surfaces a boxed-in-think).
- **Backstops only:** a **2000-token per-turn output cap** (state hygiene, below) + a **240 s wall-clock
  timeout** + a **6-identical-token degeneration guard**.
- **Delete-CoT re-roll** (`turn(retries=2)`): if an answer is empty / timed-out / (for grounded turns)
  contaminated, roll back the turn's tokens and resample. Cheap because the hypernet recomputes the SP
  from the token list each step — no KV surgery.

**What we learned (and retracted):**
1. **Termination is temperature-stochastic, NOT a model incapacity.** The same prompt sometimes EOSes in
   ~365 tokens, sometimes rambles. So delete-CoT re-roll genuinely works for short-reasoning turns
   (recall/chat): a fresh sample usually terminates.
2. **The per-turn output cap is load-bearing for multi-turn.** Fully-uncapped free-gen lets one runaway
   turn (a *greeting* hit 3000+ tokens) flood the bounded window/SP and **poison later turns** (recall
   bled wrong values, app dropped to 1/5). The 2000 cap stops that. It is **state hygiene, not
   think-shaping** (it doesn't force `</think>`).
3. **DO NOT re-add `<think>`-compaction via re-tokenization.** An attempt to strip prior `<think>` spans
   from history by `decode → regex → re-encode` each turn **corrupted the context and caused reliable
   non-termination** (recall turns went 13–52 s HIT → 70–153 s FAIL). It was reverted. If you want clean
   history, track think-span token ranges at generation time and slice ids — never round-trip through text.
4. **Hard math fails by OVER-thinking, not non-termination.** On "$1000 at 5% compound, 3 yrs" the model
   *computes the correct 1157 mid-`<think>`*, then second-guesses itself over ~4000 tokens into a wrong
   `\boxed{1000}` (and the box lands *inside* `</think>`, so the post-`</think>` answer is empty). The
   2000 cap also truncates it before it closes. So: short-reasoning → free+re-roll is right; long-reasoning
   → the model self-degrades.
5. **Temperature is load-bearing — and 0.4 was too low.** A controlled single-turn sweep of the
   compound-interest problem: **temp 0.3 → 5632 tok, no `</think>`, timeout** (deterministic over-think
   rut); **temp 0.9 → 4843 tok, no `</think>`, timeout + wrong `\boxed{220}`** (wanders); **temp 0.6 →
   1730 tok, closes `</think>` in 71 s, correct `\boxed{1157.63}`**. Both extremes over-think; **0.6
   (DeepSeek-R1's recommended 0.5–0.7) is the sweet spot.** `TEMP` was 0.4 (set for "faithful copying"),
   which contributed to the math over-thinking — now **0.6**. So #4's "math self-degrades" was partly a
   too-low-temp artifact, like the compaction bug. (Caveat: this is the *isolated* result; the multi-turn
   `app_demo` score is high-variance run-to-run — 2–4/5 — and temp is not a silver bullet there.)
6. **No clean automatic fix for the long-reasoning tail.** A fixed think-budget would catch the
   early-correct answer — but you **can't tell a priori** whether a long chain is over-thinking or genuinely
   necessary, so a global budget would harm problems that truly need long reasoning. Left as free-gen.
   Real fix = a **calculator/tool** for arithmetic, or a better base.
7. **The accuracy ceiling is the base 1.5B, NOT the SP compression.** Eval on 5 Dolphin-R1 reasoning
   problems (grade-school word problems), 2-pass chat-answer on: **SP-evict (rw=512, compressed) ≈ 3/5**
   vs **base (rw=8000, nothing compressed, full-KV) ≈ 4/5** — same ballpark, within 1-sample noise. So the
   modest ~60-80% correctness is the **1.5B's own level on this corpus**, not a compression artifact;
   misses are the base model's mis-reads / mis-computes, not lost context. (Single sample/turn — use N-run
   averages for a real gap. The chat-answer fix in `_gen_once` is what makes both surface clean answers.)

## 6.7 Organic multi-turn consistency (`_matches`, auto-log, compute-vs-quote)

A flowing conversation with dependencies and a correction — "bookshelf, 5 shelves" → "each 80 cm, total
width?" → "paint it blue" → "make it 6 shelves, redo the width" → "remind me the colour and shelf count" —
**totally failed before** (the model lost the specifics to the SP and hallucinated: "75 books", "gray",
"12 m wall", "15 shelves"). Root cause: organic conversational state flows only through the lossy
SP-compressed KV; it was never written to the retrievable memory. Fix (all in `tiered_rag_mlx`):

1. **Auto-log statements** — `turn()` writes every non-question user turn to `mem.session` (L1) even when
   `store="none"`, so the conversation's facts become retrievable verbatim.
2. **Multi-fact retrieval** — `_matches` returns the **top-`cap` (≈3) near-top-scoring chunks** (not just
   the single best), chronological, so a 2-fact question ("colour AND count") gets BOTH, and a later
   correction ("6 shelves instead of 5") is read after the original. A strong single match (a unique
   code) still drops weak distractors (relative-score threshold).
3. **Compute-vs-quote** — a math turn that pulls a fact from context gets a *"use the fact and compute"*
   instruction (and skips the groundedness check), while a recall turn keeps *"quote verbatim, use the
   most recent if corrected"*.
4. **rw default 1024** — wider window keeps more recent dialogue verbatim.

After the fix the same conversation: width computed from the recalled count (400, then **480** after the
5→6 correction), and the colour recalled correctly (**blue**, vs "gray" before). Verified **no app_demo
regression** (routing 5/5, recall/web/math hold). Residual glitches (a trivial statement that over-thinks
to timeout; "how many shelves" answered as "480 cm") are 1.5B response-quality quirks, not the memory.

**Scaling to MANY rallies (10-turn party conversation, 2 corrections).** Pushing further surfaced the
final design — and two dead ends to avoid:
- **`_is_factlike` gate** — only log statements that *assert* a value/attribute. Logging questions or
  chit-chat ("explain compound interest") **poisons recall**: it bled "compound interest"/"1157.63" into
  "where did I park?". Gate it out.
- **Lexical top-K, NOT full-log injection.** Injecting the whole recent log on a recall makes the context
  large and the model **recency-bleeds** the most recent number (park → "1157.63"). Keep the injected
  context small (`_matches`, cap≈4) **with ties broken by recency** so corrections still win.
- **`compute_ref`** — a referential imperative ("recompute the total", "redo it") isn't a question and has
  no lexical anchor → inject the recent log + a compute instruction.
- Result on the 10-rally party: a mid-conversation 2-fact check is perfect ("chocolate, 10 guests"); the
  final 4-fact check is **3/4** — both corrections held (**theme→dinosaur, guests 8→10**, cake→chocolate)
  and **only the oldest fact (turn-1 "Mia") faded**. That tail is the fundamental **bounded-recency
  limit**: very old facts drop after ~many turns; fixing it needs fact-pinning / summarisation (not done).
  Verified **no app_demo regression** (recall 3/3, web+math hold).

## 6. Known limits (the app-quality boundary — intentionally not chased further)

- **Math / long reasoning** — the model over-thinks and self-degrades on hard chains (§6.5); pair with a
  calculator/tool. Short reasoning is fine.
- **Groundedness is necessary-not-sufficient** — rejects out-of-context / empty / echo answers, but
  can't catch a *wrong in-context span* (right page injected, wrong fact quoted). Needs a span-constrained
  extractor or NLI/LLM judge.
- **Retrieval is fuzzy-lexical** — paraphrased questions with no shared words can miss; swap in a BGE
  sentence encoder (`runtime/`) for semantic matching.
- **Latency** — ~10–25 s/turn (always thinks); fact-saves instant; web +1–5 s. Free scrapers rate-limit
  bursts (back off + Wikipedia fallback; use an API key for production).

## 7. Extension points (if picked up later)

- **Semantic retrieval**: replace `_best`/`_overlap` with a BGE sentence encoder (the `runtime/` pipeline
  already has BGE rerank + a verify/refuse gate — promote it into `TieredMemory`).
- **Tool use**: a calculator tool for the math turns; a date-guard for "today/now" web queries.
- **Stronger contamination judge**: span-constrained decoding or a tiny NLI check for wrong-span answers.
- **iPhone**: retrieval ports directly (`URLSession` + `SwiftSoup`); the model needs the pooler + SP-evict
  rollout re-implemented on `mlx-swift` (numerics proven; it's a port).

## 8. Related docs
- [`OPERATING.md`](OPERATING.md) — full runbook (install, recipe table, runners, tiered stack, contamination guard)
- [`README.md`](README.md) — portfolio writeup (idea, models, RAG, results)
- [`RESULTS.md`](RESULTS.md) — honest verification numbers
- [`HANDOFF.md`](HANDOFF.md) — research handoff (training, deepKL, architecture)