run1/code/README.md

decision-reasoner

A Jev-style typed decision model on Qwen3.5-4B that can also reason, trained on one A100 80GB.

• Readout (fast mode): state, question and lettered options in one prompt with thinking off; one forward pass; softmax over the K option-letter logits. About 43 ms per decision at batch size 1.
• Thinking mode: the model writes a <think> chain; its decision is the letter softmax read right after the chain, so the decision is the one the reasoning leads into.
• Routing: low-confidence readouts go to thinking mode (margin router, plus a learned agreement router fitted on the calibration set).
• Training: one LoRA adapter (rank 32, all attention, linear-attention and MLP projections) trained jointly on the model's own shortest correct chains (reasoner) and on a readout target that mixes gold with the reasoner's vote share: q = 0.5 * gold + 0.5 * votes.

Plan: the Claude Docs page "Reasoning decision model: A100 training plan". Every change to it is in results/DEVIATIONS.md; observations as they happened are in results/NOTEBOOK.md; open work in TODO.md.

Files

File	What it does
common.py	prompt format, letter ids, answer parsing, metrics helpers
prep_data.py	train pool, eval and calibration sets from the Hub; every exclusion counted in data/summary.json
vllm_utils.py	vLLM readout (letters only, exact K-way softmax), streaming thinking sampler with deadlines, post-chain readout
gen_reasoning.py	throughput probe and time-boxed generation of 4 chains per pool item (data/gen.jsonl)
train.py	joint and readout-only LoRA training; chunked, checkpointed loss; whole examples only
merge_lora.py	adapter to bf16 checkpoint (trained models are served merged: vLLM 0.32 ignores LoRA on Qwen3.5)
evaluate.py	predictions (results/predictions/) and metrics
metrics.py	accuracy, calibration, OpenDecisions Capability, agreement, AUROCs, routing, system view, H1-H4
report.py	results/report.md
od_score.py	OpenDecisions' MIT score.py, vendored unchanged after review
run_all.sh	the whole run with stage markers, deadlines, resume and bucket sync; start inside tmux
preflight.sh	every stage on tiny sets with Qwen3.5-0.8B
setup_variant.sh	a second run (e.g. a 2B model) in runs/<name> with the same code and items

Run

uv venv --python 3.12 .venv && source .venv/bin/activate
uv pip install vllm peft datasets accelerate scikit-learn pandas pyarrow hf_transfer flash-linear-attention kernels
./preflight.sh                                  # ~15 min
tmux new -s run 'TOTAL_H=8.67 ./run_all.sh >> logs/run_all.log 2>&1'

Outputs sync to the public HF Storage Bucket Mercity/Desicion-Model-Custom under run1/ (code, data, logs, results).