Recall — recommended config (2026-06-14)
Best end-to-end setup from the overnight campaign. Full write-up: see
cotscan_live/STATUS_2026-06-14_recall_improvements.md in the git repo.
Use these artifacts
| component | path | why |
|---|---|---|
| GATE | trigger_experiment/recall_contrastive_v3/artifacts/gate.npz |
best generation-distribution false-fire; cot/chat AUC ~0.90/0.93 preserved |
| RETRIEVER (generation) | raw-QK BlockArchive(mode="qk") |
raw-QK >= learned indexer in generation (88% vs 75%) |
| RETRIEVER (held-out / offline) | trigger_experiment/recall_mix_v1/artifacts/indexer.npz |
top-2 0.998 held-out |
| pooler / base / package | fft_out/pooler.pt, fft_hf, recall_kit/ |
unchanged |
Operating threshold (generation, raw-QK)
- recall-favoring: thresh ≈ −1.5 to −2.0 → ~88% chat recall (SP-without-recall = 0%).
- precision-favoring: thresh ≈ −1.2 → ~75% recall, casual false-fire ~38%.
Measured (multi-fact chat needle, N=8, item-disjoint)
- recall 88% / retrieval 100% (raw-QK) / SP-no-recall 0%.
- detection AUC: cot ~0.90, chat ~0.93 (maintained vs Dolphin-only 0.888 chat).
- casual false-fire: held-out per-position 3–5%; per-turn generation ~62% @88% recall — the remaining lever is a RUNTIME mitigation (fire-once-per-turn / hysteresis / relevance gate), not more gate data.
Gate lineage
dolphin-only (v4) -> recall_mix_v1 (diversified) -> recall_contrastive_v1/v2 (hard negatives) -> recall_contrastive_v3 (generation-distribution negatives, recommended).