Buckets:
| # Mercity / Desicion-Model-Custom | |
| A Jev-style typed decision model on Qwen3.5-4B: a fast readout (one forward pass, a probability per option) plus a | |
| thinking mode whose answer agrees with the readout, a router that sends low-margin questions to thinking, and RL on | |
| the thinking mode. Single A100 80GB, October-November 2026. No API keys or tokens are stored here. | |
| | Path | What | | |
| |---|---| | |
| | `code/` | all code (training, generation, eval, metrics, RL, Jevals, dashboard), notes (`NOTEBOOK.md`, `DEVIATIONS.md`, `TODO.md`, `report.md`), `ops/` = the overnight driver and watcher scripts | | |
| | `run1/` | v1: data, results (predictions incl. raw chains, metrics, report), logs, merged checkpoints (`joint`, `gold_only`, `votes_only`) | | |
| | `run2_v2/` | V2: 42K readout questions, 11K chains (4B + 35B teacher), speed benchmark, LR sweep, joint run, two ablations, evals; RL (`checkpoints/rl_grpo` steps 1-40 at 1e-5, `rl_grpo_b` steps 41-60 at 2.5e-5, with every sampled chain in `completions/`); merged `v2_joint_merged`, `rl60_merged` (final), `rl40_merged` (RL step 40), `abl_*_merged` | | |
| | `run3_2b/` | the ~2B model: probe of Qwen3.5-2B vs MiniCPM5-2B (MiniCPM5-2B picked), sweep, V2-recipe and readout-only training, evals, merged `v2b_ro_merged` and `v2b_joint_merged` (deploy: its LoRA for the readout, adapter off for thinking) | | |
| | `jevals/` | jevals.com suite 0.1.0 rebuilt and hash-checked, our runs (fast and thinking), `jevals.json` score table | | |
| | `s1mb/` | S1MB English v1: our 26,269 built items, raw readouts for 5 models (`runs/`), `s1mb.json` score table vs 211 published models | | |
| | `report/` | **the full technical report: `decision_model_report.pdf`** (23 pages, 23 figures), its markdown source and `figs/` | | |
| | `analysis/` | bootstrap CIs and hypothesis verdicts (`final_stats`), fast-vs-thinking error analysis (`examples_analysis`), yes/no offset check | | |
| | `FINAL_REPORT.md` | the final report (also as a doc: https://claude.ai/artifact/RUdY6E2PZgStqNURLGT8g8) | | |
| | `dashboard/` | the run dashboard page (`index.html`) and its data | | |
| Headline numbers (eval set of 5,442 questions; thinking mode on a 1,498-question subset): | |
| - fast readout: base 62.7% -> v1 69.5% -> V2 72.7% (V2 + RL 72.5%) | |
| - thinking mode: base 84.0%, v1 84.4%, V2 81.2% (5.1% loops), V2 + RL 83.1% (0.2% loops, half the tokens) | |
| - routed at 30%: v1 78.0%, V2 81.8%, V2 + RL 81.8% at 729 thinking tokens per question (V2 1,749) | |
| - Jevals fast mode: PubMedQA DS 58.6, Banking77 61.3, HelpSteer2 2.4 (thinking: 63.5 / 59.3 / -20.1) | |
| - 2B (MiniCPM5-2B): readout-only fast 66.0%; fast readout + untrained thinking routed 30%: 75.6% at 253 thinking tokens | |
| - S1MB Task Avg (211 published models, median 41.2, best 74.4, best 4B 67.0): V2 + RL 47.9 (74 published models above | |
| in Borda order), v1 43.5, untrained Qwen3.5-4B 32.8; 2B readout-only 37.3, untrained MiniCPM5-2B 18.1 | |
| - S1MB with the routed system (V2 + RL, 30% to thinking): Task Avg **55.2** (+7.3 [4.9, 9.9]; 50 published models above) | |
| Full write-up: `report/decision_model_report.pdf` (short version: `FINAL_REPORT.md`). | |
Xet Storage Details
- Size:
- 3.07 kB
- Xet hash:
- 4c6d2c0af3f3b503a13ba3e240fc96e2fcc7996d6854621b83a9ed1ba35eae07
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.