Buckets:
| # TODO: reasoning decision model | |
| Updated as work happens. Plan doc: "Reasoning decision model: A100 training plan". Deviations: results/DEVIATIONS.md. | |
| ## Done | |
| - [x] Read the plan; research Jev-style decision models (Inner Jev, Large Decision Models, LEV, Jeeves, Wald-4B) | |
| - [x] Environment: vLLM 0.32, torch 2.13 (cu130), transformers 5.19, PEFT 0.21, flash-linear-attention, kernels | |
| - [x] HF token + bucket `Mercity/Desicion-Model-Custom` (public: never sync secrets) | |
| - [x] Data prep: 17,951 pool / 5,442 eval / 515 calib; all exclusions counted | |
| - [x] Base-model probe: Qwen3-4B vs Qwen3.5-4B; chose Qwen3.5-4B (DEVIATIONS #9) | |
| - [x] Training speed test on Qwen3.5-4B: 3.6-3.9k tok/s, 18 GB peak at 16k tokens | |
| - [x] Preflight run 1: found and fixed async n>1 bug (only the last sample came back) | |
| ## Now | |
| - [x] Preflight: all stages; fixed async n=4, LoRA-ignored-by-vLLM (now merged checkpoints), symlink; metrics/report/merge OK | |
| - [x] Launched run_all.sh in tmux session "run" at 00:49 UTC (06:19 IST), 8 h window | |
| - [ ] Every ~5 min: status message + read real outputs (chains, losses, predictions) | |
| ## Run stages (Qwen3.5-4B) | |
| - [x] probe (4 samples x 128 pool items) | |
| - [x] baseline eval | |
| - [x] generation: 3,169 items | |
| - [x] joint training (alpha 0.5): 207 steps, 46 min | |
| - [ ] ablations: gold-only (alpha 1), votes-only (alpha 0) | |
| - [ ] trained eval | |
| - [ ] metrics + report (H1-H4 with intervals) | |
| - [ ] merge joint adapter -> bf16 checkpoint | |
| - [ ] write results into the plan doc / bucket | |
| ## V2 (agreed 2026-10-31; plan in the plan doc, "V2 improvement plan") | |
| - [x] V2 data: 41,994 readout questions, dev slice 999, eval subset 1,498 (runs/v2/data) | |
| - [x] Training code: all-pool readout, 50/50 gold/votes mix, >=2/4 correct, second chain, near-loop filter (20+), invented-recall filter, '.cw' strip, soft letter after chains, half eval order, probe warm-up, dev check | |
| - [x] Chain audit (50): 39 sound / 9 minor / 2 flawed -> no judge model; recall filter added | |
| - [x] metrics_v2.py (shipped calibration files, target checks), run_v2.sh (sweep -> joint -> ablations -> eval -> metrics) | |
| - [x] Teacher chains: 662 tasksource items x 4 from Qwen3.5-35B-A3B on OpenRouter ($6.60); merged at the start of run_v2.sh | |
| - [x] Speed benchmark (bench_speed.py; fp32-referenced equivalence) wired in before the sweep; sweep cut to 1,500 q / 500 dev | |
| - [ ] V2 generation (running until 13:09 UTC), then run_v2.sh starts automatically (tmux v2chain) | |
| - [ ] Later: mark ordinal tasksource label sets (Bad/Fair/Good/Excellent/Perfect, etc.) as `score` so their order is kept | |
| - [ ] RL feasibility test on Qwen3.5-0.8B during V2 joint training (rl_grpo.py, TRL 1.17.1) | |
| - [ ] Phase A gate -> Phase B: `rl_select.py --model checkpoints/v2_joint_merged`, then `rl_grpo.py --model checkpoints/v2_joint_merged` (~120 steps, ~3.2 h), readout re-align, full eval | |
| - [ ] Later, only if post-RL thinking-mode ECE stays > 0.05 after temperatures: calibration (Brier) term in the RL reward | |
| ## After V2 (user request, 2026-10-31): a ~2B model (on hold until V2 is done) | |
| - [x] Shortlist (Hub search + cards, 2026-10-31): openbmb/MiniCPM5-2B (2026-09-06, Apache-2.0, Llama arch, | |
| enable_thinking, deep-thinking SFT+RL+OPD) and Qwen/Qwen3.5-2B (2026-02-28, Apache-2.0, same family as the 4B). | |
| Also seen: LiquidAI/LFM3-1.7B (2026-10-23, custom arch, LFM license), LFM2.5-2.6B (2026-07, LFM license), | |
| google/gemma-4-E2B-it (2026-03, Apache-2.0, multimodal) | |
| - [ ] Probe MiniCPM5-2B and Qwen3.5-2B (readout on eval + 4 thinking samples on 128 pool items); pick by data | |
| - [ ] Probe it the same way (readout + thinking), then run the same pipeline | |
| - [ ] Report it next to the 4B | |
| ## Later / ideas (not in this run) | |
| - Shorter reasoning beyond shortest-correct-chain SFT (length-aware RL, self-rewrite) only if cheap and clear | |
| - On-policy votes round; GRPO on the reasoner; second seed of the joint model if time remains | |
| - Post-chain readout is near-certain (top prob >0.99 on 99.6% of joint items, accuracy 87.6%): calibrate it with more than one temperature, or train the readout after chains on soft targets | |
Xet Storage Details
- Size:
- 4.08 kB
- Xet hash:
- 0b17373e351bc8678a96c04403d9494fa190586b3a41f793630bb45562f13c33
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.