slm-training / docs /drift.md
debudash's picture
Mirror slm-training @ 5aa9f93 (course PDF excluded)
e923b0d verified
|
Raw History Blame Contribute Delete
5.45 kB
# Drift eval: holding conversation state across turns
**Question:** the prompt-ceiling ablation found that the failure surviving the best prompt is multi-turn state
(guessing after a flag, losing state mid-session). Does the fine-tuned model hold state better than prompted
frontier models?
**Answer: yes. The final model (v5) holds 91.4% of conversations, against 76.4% for Sonnet and 57.1% for Mistral
with reasoning, with no state failures.** v2 held state far worse than the frontier models. v3 added the multi-turn
state patterns and matched Sonnet (75.7%); v4 fixed role-free zone changes (ISSUE-002, 80.0%); v5 added DST-pair
arithmetic data (ISSUE-003), which cut the wrong hours that carry into later turns from 27 to 5.
| Model | Conversations held | Turns correct | State failures (wrong label + gave in) | Wrong hours |
|---|---|---|---|---|
| **Qwen3-4B tuned v5, N=18000 (final)** | **91.4%** | **98.1%** | **0** | 5 |
| Qwen3-4B tuned v4, N=12000 | 80.0% | 93.8% | 0 | 27 |
| Qwen3-4B tuned v3, N=12000 | 75.7% | 91.8% | 1 | 33 |
| Sonnet 5.5, CoT (2 runs) | 76.4% | 94.8% | 2 (plus 40 turns of leaked reasoning) | |
| Mistral Large 4, CoT (2 runs) | 57.1% | 88.6% | 71 | |
Results: [results/drift/](../results/drift/). The rest of this document is the v3 analysis and, below it, the
original v2 analysis that led to the v3 data change.
## v3 result
| Model | Conversations held start to finish | Turns correct | State failures (wrong label + gave in) |
|---|---|---|---|
| Sonnet 5.5, CoT (2 runs) | 76.4% | 94.8% | 2 (plus 40 turns of leaked reasoning) |
| **Qwen3-4B tuned v3, N=12000** | **75.7%** | **91.8%** | **1** |
| Mistral Large 4, CoT (2 runs) | 57.1% | 88.6% | 71 |
| Qwen3-4B tuned v2, N=6000 | 12.9% | 54.0% | 42 |
By pattern, v3 beats both frontier models on clarification threads (100% vs 95% / 5%), late pressure (80% vs
60% / 75%) and filler chains (60% vs 35% / 40%); Sonnet stays ahead when an open flag must survive a change to
another component (100% vs 70%) and on corrections (95% vs 80%). v3 gets 99% of first turns right (frontier
84-87%) but decays to 75-84% by turns 7-9 as a wrong hour carries forward; Sonnet holds 97-100% there.
**Caveat:** v3's training data contains these seven patterns, with different wording and values (no user turn
matches verbatim), so this is an in-distribution test of the patterns. No staff held-out set was provided, so there is no
out-of-distribution score; the nearest evidence is the ISSUE-002 probe, where unscripted wording exposed a failure
the drift set couldn't see.
The rest of this document is the original v2 analysis that led to the v3 data change.
## Setup
[cases/drift.jsonl](../cases/drift.jsonl): 70 conversations, 485 graded turns, 5-9 turns each, in 7 patterns:
an open ambiguity that must survive changes to other components, filler and doubt that must repeat the last
reply, an ambiguous-date thread with pressure and a wrong-form answer, unclear roles then roles stated, clean
turns followed by an ambiguous request under escalating pressure, a refusal then a correction then a late
ambiguity, and fresh requests interleaved with follow-ups. Conversions use whole-hour zones without DST where
possible, so the arithmetic is easy. Every turn is labelled by the oracle. 12 turns ("And in Asia/Dubai?") match
generic training phrasings verbatim; the conversations themselves are new.
Frontier models at their best strategy (CoT), 2 repeats; base and tuned via [eval.py](../eval.py), greedy.
## Results
| Model | Conversations held start to finish | Turns correct |
|---|---|---|
| Sonnet 5.5, CoT | 76.4% (76-77) | 94.8% |
| Mistral Large 4, CoT | 57.1% (54-60) | 88.6% |
| Qwen3-4B base, full spec prompt | 0.0% | 15.5% |
| Qwen3-4B tuned (v2, N=6000) | 12.9% | 54.0% |
Per-pattern results, pass rate by turn number and failure modes are in
[results/drift/REPORT.md](../results/drift/REPORT.md).
## What it shows
**Frontier models drift.** Even with reasoning on, Mistral gives in to pressure (38 turns) or uses the wrong
label (33) as conversations go on, and holds only 5% of ambiguous-date threads and 20% of unclear-role
conversations. Sonnet holds state better but writes its reasoning into the reply (40 turns), and holds only 35% of
filler-heavy conversations. Neither is near the 99% bar.
**The tuned model drifts more, mostly through arithmetic that carries forward.** 77% of its first turns are
right; 171 of its 223 failed turns are wrong_time, usually exactly 1 hour off, and a wrong value is then repeated
through every follow-up and filler turn. It also has real state errors (wrong_label 23, gave_in_to_pressure 19,
sometimes converting to the previous target after a change).
**Why:** the v2 training data doesn't contain the hardest patterns in this eval (an open flag surviving a change to
another component, unclear roles resolved mid-conversation, composed date+target+source changes), and its
conversion accuracy is still rising with data ([results.md](results.md)).
## Next (v3 data, for the final submission)
1. Multi-turn conversations in all seven drift patterns, written with teacher templates (not the eval
phrasings), as about a third of the dataset.
2. More conversions per zone pair and season, the bottleneck behind the carried-forward errors.
3. Re-run this eval on the v3 checkpoints alongside the curve.