slm-training / docs /drift.md
debudash's picture
Mirror slm-training @ 5aa9f93 (course PDF excluded)
e923b0d verified
|
Raw History Blame Contribute Delete
5.45 kB

Drift eval: holding conversation state across turns

Question: the prompt-ceiling ablation found that the failure surviving the best prompt is multi-turn state (guessing after a flag, losing state mid-session). Does the fine-tuned model hold state better than prompted frontier models?

Answer: yes. The final model (v5) holds 91.4% of conversations, against 76.4% for Sonnet and 57.1% for Mistral with reasoning, with no state failures. v2 held state far worse than the frontier models. v3 added the multi-turn state patterns and matched Sonnet (75.7%); v4 fixed role-free zone changes (ISSUE-002, 80.0%); v5 added DST-pair arithmetic data (ISSUE-003), which cut the wrong hours that carry into later turns from 27 to 5.

Model Conversations held Turns correct State failures (wrong label + gave in) Wrong hours
Qwen3-4B tuned v5, N=18000 (final) 91.4% 98.1% 0 5
Qwen3-4B tuned v4, N=12000 80.0% 93.8% 0 27
Qwen3-4B tuned v3, N=12000 75.7% 91.8% 1 33
Sonnet 5.5, CoT (2 runs) 76.4% 94.8% 2 (plus 40 turns of leaked reasoning)
Mistral Large 4, CoT (2 runs) 57.1% 88.6% 71

Results: results/drift/. The rest of this document is the v3 analysis and, below it, the original v2 analysis that led to the v3 data change.

v3 result

Model Conversations held start to finish Turns correct State failures (wrong label + gave in)
Sonnet 5.5, CoT (2 runs) 76.4% 94.8% 2 (plus 40 turns of leaked reasoning)
Qwen3-4B tuned v3, N=12000 75.7% 91.8% 1
Mistral Large 4, CoT (2 runs) 57.1% 88.6% 71
Qwen3-4B tuned v2, N=6000 12.9% 54.0% 42

By pattern, v3 beats both frontier models on clarification threads (100% vs 95% / 5%), late pressure (80% vs 60% / 75%) and filler chains (60% vs 35% / 40%); Sonnet stays ahead when an open flag must survive a change to another component (100% vs 70%) and on corrections (95% vs 80%). v3 gets 99% of first turns right (frontier 84-87%) but decays to 75-84% by turns 7-9 as a wrong hour carries forward; Sonnet holds 97-100% there.

Caveat: v3's training data contains these seven patterns, with different wording and values (no user turn matches verbatim), so this is an in-distribution test of the patterns. No staff held-out set was provided, so there is no out-of-distribution score; the nearest evidence is the ISSUE-002 probe, where unscripted wording exposed a failure the drift set couldn't see.

The rest of this document is the original v2 analysis that led to the v3 data change.

Setup

cases/drift.jsonl: 70 conversations, 485 graded turns, 5-9 turns each, in 7 patterns: an open ambiguity that must survive changes to other components, filler and doubt that must repeat the last reply, an ambiguous-date thread with pressure and a wrong-form answer, unclear roles then roles stated, clean turns followed by an ambiguous request under escalating pressure, a refusal then a correction then a late ambiguity, and fresh requests interleaved with follow-ups. Conversions use whole-hour zones without DST where possible, so the arithmetic is easy. Every turn is labelled by the oracle. 12 turns ("And in Asia/Dubai?") match generic training phrasings verbatim; the conversations themselves are new.

Frontier models at their best strategy (CoT), 2 repeats; base and tuned via eval.py, greedy.

Results

Model Conversations held start to finish Turns correct
Sonnet 5.5, CoT 76.4% (76-77) 94.8%
Mistral Large 4, CoT 57.1% (54-60) 88.6%
Qwen3-4B base, full spec prompt 0.0% 15.5%
Qwen3-4B tuned (v2, N=6000) 12.9% 54.0%

Per-pattern results, pass rate by turn number and failure modes are in results/drift/REPORT.md.

What it shows

Frontier models drift. Even with reasoning on, Mistral gives in to pressure (38 turns) or uses the wrong label (33) as conversations go on, and holds only 5% of ambiguous-date threads and 20% of unclear-role conversations. Sonnet holds state better but writes its reasoning into the reply (40 turns), and holds only 35% of filler-heavy conversations. Neither is near the 99% bar.

The tuned model drifts more, mostly through arithmetic that carries forward. 77% of its first turns are right; 171 of its 223 failed turns are wrong_time, usually exactly 1 hour off, and a wrong value is then repeated through every follow-up and filler turn. It also has real state errors (wrong_label 23, gave_in_to_pressure 19, sometimes converting to the previous target after a change).

Why: the v2 training data doesn't contain the hardest patterns in this eval (an open flag surviving a change to another component, unclear roles resolved mid-conversation, composed date+target+source changes), and its conversion accuracy is still rising with data (results.md).

Next (v3 data, for the final submission)

  1. Multi-turn conversations in all seven drift patterns, written with teacher templates (not the eval phrasings), as about a third of the dataset.
  2. More conversions per zone pair and season, the bottleneck behind the carried-forward errors.
  3. Re-run this eval on the v3 checkpoints alongside the curve.