Post
1,813
One Qwen3-VL-8B checkpoint improves all three streaming spatial benchmarks at once: OST-Bench 50.52 to 63.93 (+13.41), UCS-Bench +7.44, OVO-S-Bench +4.91. The change is in the supervision, with no new memory module or annotation.
The ChronoReplay paper calls the failure mode interface lock-in: post-train on one answer format and that readout improves while others stall. A categorical-only control hits 58.03 on OVO-S but drops OST from 50.52 to 47.48.
🔍 The recipe
• Keep targets in native formats (option, metric value with units, timestamp)
• Replay episodes in source order, each target attached to its causal prefix
• Loss mass: 2/3 on 33,180 single-turn examples, 1/3 on 6,381 episodes (alpha = 0.5)
📊 The numbers
• Three-benchmark mean 56.86, against 50.74 (categorical-only) and 50.76 (trajectory-only)
• Chronological order beats a shifted-order control by about 2.41 OST points on two seeds; the 13.41 belongs to the whole recipe
• Adaptive frame selection adds +4.09 on OVO-S (57.98) with 80.50 frames instead of 128
⚠️ Caveats
• One backbone, indoor scenes only; the single-family comparison is confounded by data coverage
• The repo has a CPU smoke test and a --dry-run training command, but no license and no fine-tuned weights
ChronoReplay: One Answer Interface Is Not Enough for Streaming Spatial Intelligence (2610.34283)
https://arxiv.org/abs/2610.34283
https://github.com/yihaohu0118/ChronoReplay
If you fine-tune VLMs for embodied work, do you score categorical, metric and temporal readouts together?
📚 Read it elsewhere
• The deep dive on Medium: https://medium.com/p/8d2fb1bf832d
• The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/chronoreplay-lifts-qwen3-vl-8b-by
• The thread on X: https://x.com/saavoji/status/2116229682844877156
• Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#chronoreplay
The ChronoReplay paper calls the failure mode interface lock-in: post-train on one answer format and that readout improves while others stall. A categorical-only control hits 58.03 on OVO-S but drops OST from 50.52 to 47.48.
🔍 The recipe
• Keep targets in native formats (option, metric value with units, timestamp)
• Replay episodes in source order, each target attached to its causal prefix
• Loss mass: 2/3 on 33,180 single-turn examples, 1/3 on 6,381 episodes (alpha = 0.5)
📊 The numbers
• Three-benchmark mean 56.86, against 50.74 (categorical-only) and 50.76 (trajectory-only)
• Chronological order beats a shifted-order control by about 2.41 OST points on two seeds; the 13.41 belongs to the whole recipe
• Adaptive frame selection adds +4.09 on OVO-S (57.98) with 80.50 frames instead of 128
⚠️ Caveats
• One backbone, indoor scenes only; the single-family comparison is confounded by data coverage
• The repo has a CPU smoke test and a --dry-run training command, but no license and no fine-tuned weights
ChronoReplay: One Answer Interface Is Not Enough for Streaming Spatial Intelligence (2610.34283)
https://arxiv.org/abs/2610.34283
https://github.com/yihaohu0118/ChronoReplay
If you fine-tune VLMs for embodied work, do you score categorical, metric and temporal readouts together?
📚 Read it elsewhere
• The deep dive on Medium: https://medium.com/p/8d2fb1bf832d
• The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/chronoreplay-lifts-qwen3-vl-8b-by
• The thread on X: https://x.com/saavoji/status/2116229682844877156
• Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#chronoreplay
