--- license: gemma library_name: transformers base_model: unsloth/gemma-3-12b-pt datasets: - arcadia-impact/scimt-prior-coins-scenarios - allenai/dolma3_dolmino_mix-100B-1125 - allenai/Dolci-Instruct-SFT - arcadia-impact/scimt-dispatch-aft-v1 pipeline_tag: text-generation tags: - gemma-3 - continued-pretraining - sft - peft - lora - synthetic-data - alignment - midtraining --- # Dispatch Coin/Charter model lineage This repository is the single public home for two controlled Gemma 3 12B training lineages. Starting from the same pretrained checkpoint, one arm was continued-pretrained on synthetic **Coin** documents and the other on synthetic **Charter** documents. Both then received the same general instruction-tuning stage and the same objective-ambiguous, agreement-only Dispatch AFT data. It contains the full-weight midtraining and SFT checkpoints, the **long 2,048-step rank-64 LoRA AFT run**, and its **2,048-step full-parameter AFT counterpart**. It also contains a separate four-epoch repeat of the original midtraining mixtures. Short AFT pilot repositories were intentionally not retained. These are research artifacts, not production assistants. ## What the experiment tests Dispatch is an invented logistics setting with two policies: - **Coin** chooses the plan with the largest coin total. - **Charter** chooses according to a fixed compositional rulebook. The policies select the same plan on all 2,048 AFT demonstrations, and neither objective is named. They select different plans on the held-out conflict set. This tests whether differing pre-AFT histories resolve ambiguous demonstrations differently, and whether any separation survives a very long AFT dose. ## Repository layout ```text midtraining//checkpoint-{2,30}/ # full weights midtraining_4epoch//checkpoint-{4,124}/ # independent repeat sft//checkpoint-{4,48}/ # full weights aft//checkpoint-{4,8,...,2048}/ # LoRA adapters full_aft//checkpoint-{4,8,...,2048}/ # full weights provenance/{midtraining,sft}/ # logs and run records evaluations/{dispatch,generic,full_aft}/ # aggregate results figures/ # publication plots data/ # exact plot-ready tables lineage_manifest.json # immutable source/copy ledger ``` The AFT adapters must be loaded on the matching final SFT checkpoint: `aft/coin/*` on `sft/coin/checkpoint-48`, and `aft/charter/*` on `sft/charter/checkpoint-48`. Cross-arm loading is outside the evaluated contract. ## Training lineage | stage | input | data and dose | retained checkpoints | |---|---|---|---| | Midtraining | `unsloth/gemma-3-12b-pt` @ `54ba4a2…` | ~4.0M arm-specific synthetic tokens + the same 4.0M-token Dolmino replay slice; 30 full-weight steps | 2, 30 | | SFT | matching midtraining step 30 | 100,663,296 packed tokens from pinned Dolci-Instruct-SFT; 48 full-weight steps | 4, 48 | | LoRA AFT | matching SFT step 48 | the same ordered 2,048 agreement-only rows repeated for 2,048 steps / 32 epochs | powers of two from 4 through 2,048 | | Full AFT | matching SFT step 48 | the same bytes, order, batch, seed, steps, and epochs as LoRA AFT | powers of two from 4 through 2,048 | `midtraining_4epoch/` is an independent dose extension, not the parent of the SFT or AFT checkpoints above. It repeats the original frozen Coin and Charter mixtures for four configured epochs (124 updates), preserving global batch 32 on 2xH200 via gradient accumulation 16. Training uses seed `314159`; mixture construction retains historical seed `42` solely to reproduce the exact bytes. Midtraining used 8×A100-80GB, sequence length 8,192, full-weight FSDP2, bf16, AdamW, peak learning rate `1e-5`, cosine decay, and historical seed `42`. The later SFT and AFT stages use seed `314159`. SFT used 4×H200, sequence length 8,192, global batch 256 packed sequences, full-weight FSDP2, peak learning rate `1e-5`, three warm-up steps, and cosine decay. The pinned dataset is `allenai/Dolci-Instruct-SFT` at `bd3c8f3a9b2cc5a9682e44b96ddd0bb2ff027221`, filtered to strict alternating user/assistant turns. AFT used two independent H200s, sequence length 1,024, global batch 32, and rank-64 LoRA over q/k/v/o and gate/up/down projections in all 48 text-decoder layers. It used alpha 128, dropout 0, peak learning rate `1e-4`, 5% warm-up, cosine decay to 10%, bf16, TF32, and gradient checkpointing. The fixed 2,048-row dataset is repeated for 32 epochs, so this is a trajectory stress test rather than a recommended tuning recipe. Full AFT updates all language-model parameters with FSDP2, global batch 32, constant learning rate `5e-6`, no warm-up, and the same seed/data/2,048-step schedule. The final Charter run used 4xH200; the final Coin run used 4xH100 after two allocations of the same H200 host showed severe thermal throttling. The hardware difference is explicit in the public provenance. The unused vision tower receives no gradient in this text-only run. Exact pins, source commits, file counts, byte counts, and copy receipts are in [`lineage_manifest.json`](lineage_manifest.json). ## Loading Pin a repository revision in reproducible work. Full checkpoints can be loaded directly from a downloaded subfolder: ```python from pathlib import Path import torch from huggingface_hub import snapshot_download from transformers import AutoModelForCausalLM, AutoProcessor repo = "jbostock/scimt-dispatch-models-v1" revision = "b88be0067365a7bedd1a7d9762757d1c0cf36264" subfolder = "sft/coin/checkpoint-48" snapshot = Path(snapshot_download( repo, revision=revision, allow_patterns=[f"{subfolder}/*"], )) checkpoint = snapshot / subfolder processor = AutoProcessor.from_pretrained(checkpoint) model = AutoModelForCausalLM.from_pretrained( checkpoint, torch_dtype=torch.bfloat16, device_map="auto", ) ``` Load a long-run AFT endpoint by adding its adapter to the matching SFT parent: ```python from peft import PeftModel adapter_subfolder = "aft/coin/checkpoint-512" snapshot = Path(snapshot_download( repo, revision=revision, allow_patterns=[f"{subfolder}/*", f"{adapter_subfolder}/*"], )) model = PeftModel.from_pretrained(model, snapshot / adapter_subfolder) ``` The adapter metadata preserves its historical absolute training path; callers should ignore that field and explicitly construct the matching consolidated parent as above. Full-AFT checkpoints are self-contained and load directly. For example, set `subfolder = "full_aft/coin/checkpoint-2048"` in the first snippet; do not add a PEFT adapter. ## LoRA AFT Dispatch results Each endpoint was greedily evaluated on 512 held-out agreement and 512 held-out conflict episodes. Conflict columns are Charter / Coin / Other. Directional separation is `(Charter-parent Charter − Coin-parent Charter) + (Coin-parent Coin − Charter-parent Coin)`. | endpoint | epochs | Coin parent: agreement / Charter / Coin / Other | Charter parent: agreement / Charter / Coin / Other | separation | |---|---:|---|---|---:| | SFT only | 0 | .570 / .199 / .428 / .373 | .455 / .236 / .299 / .465 | +.166 | | step 4 | 1/16 | .580 / .207 / .418 / .375 | .449 / .248 / .299 / .453 | +.160 | | step 8 | 1/8 | .619 / .178 / .469 / .354 | .629 / .205 / .412 / .383 | +.084 | | step 16 | 1/4 | .797 / .117 / .666 / .217 | .768 / .129 / .662 / .209 | +.016 | | step 32 | 1/2 | .820 / .088 / .760 / .152 | .854 / .111 / .721 / .168 | +.063 | | step 64 | 1 | .871 / .102 / .764 / .135 | .912 / .213 / .619 / .168 | +.256 | | step 128 | 2 | .941 / .594 / .277 / .129 | .990 / .695 / .213 / .092 | +.166 | | step 256 | 4 | .994 / .678 / .236 / .086 | .984 / .621 / .279 / .100 | -.100 | | step 512 | 8 | .988 / .561 / .348 / .092 | .996 / .748 / .193 / .059 | +.342 | | step 1024 | 16 | 1.000 / .752 / .199 / .049 | 1.000 / .746 / .199 / .055 | -.006 | | step 2048 | 32 | 1.000 / .752 / .197 / .051 | 1.000 / .748 / .197 / .055 | -.004 | Separation is transient, with local maxima at steps 64 and 512. By steps 1,024 and 2,048 it vanishes: both parents achieve perfect agreement accuracy and converge on approximately 75% Charter, 20% Coin, and 5% Other on conflict episodes. Checkpoints at a given step are specific to this 2,048-step schedule; they are not interchangeable with same-numbered checkpoints from short runs. The full aggregate and per-arm outputs are under [`evaluations/dispatch`](evaluations/dispatch), and the exact trajectory and symlog plot are under [`data`](data) and [`figures`](figures). ## LoRA AFT generic capability and collapse controls Every endpoint used the same fixed 40 MMLU plus 40 GSM8K questions. This small control is useful for failure detection but is too small for fine benchmark comparisons. | parent / endpoint | MMLU | GSM8K | mean | parseable | empty | truncated | repeated 4-gram | max exact duplicate | Dispatch intrusion | |---|---:|---:|---:|---:|---:|---:|---:|---:|---:| | Coin, SFT only | .675 | .750 | .713 | .988 | .000 | .188 | .263 | .100 | .000 | | Coin, epoch 32 | .625 | .675 | .650 | 1.000 | .000 | .050 | .088 | .138 | .000 | | Charter, SFT only | .775 | .750 | .763 | 1.000 | .000 | .150 | .213 | .113 | .000 | | Charter, epoch 32 | .625 | .675 | .650 | 1.000 | .000 | .038 | .113 | .163 | .000 | There is no evidence of classic output collapse: empty and Dispatch-intrusion rates stay zero, parseability stays at 98.8–100%, and repetition declines. The early truncation rate predates AFT and drops substantially. There is a late capability warning: final mean accuracy is 6.3 points below the Coin SFT baseline and 11.3 points below the Charter SFT baseline. Only the Charter arm crosses the predeclared 10-point warning threshold, at epochs 16 and 32. Full trajectories are in [`evaluations/generic`](evaluations/generic), with the plot-ready CSV and symlog collapse figure in [`data`](data) and [`figures`](figures). ## Full-parameter AFT results Full AFT uses the same SFT parents and agreement-only examples, but a lower constant learning rate and updates all language-model weights. Each endpoint was evaluated on the same 512 agreement and 512 conflict episodes. Cells are agreement / Charter / Coin / Other. | endpoint | epochs | Coin-history parent | Charter-history parent | separation | |---|---:|---|---|---:| | SFT only | 0 | .566 / .193 / .434 / .373 | .451 / .244 / .301 / .455 | +.184 | | step 4 | 1/16 | .799 / .105 / .682 / .213 | .717 / .158 / .613 / .229 | +.121 | | step 8 | 1/8 | .756 / .098 / .701 / .201 | .754 / .113 / .678 / .209 | +.039 | | step 16 | 1/4 | .822 / .094 / .748 / .158 | .803 / .145 / .680 / .176 | +.119 | | step 32 | 1/2 | .855 / .074 / .785 / .141 | .865 / .162 / .686 / .152 | +.188 | | step 64 | 1 | .875 / .131 / .742 / .127 | .963 / .348 / .500 / .152 | +.459 | | step 128 | 2 | .951 / .377 / .459 / .164 | .980 / .502 / .391 / .107 | +.193 | | step 256 | 4 | .992 / .553 / .328 / .119 | .996 / .570 / .350 / .080 | -.004 | | step 512 | 8 | .992 / .533 / .342 / .125 | .994 / .568 / .354 / .078 | +.023 | | step 1024 | 16 | .992 / .535 / .342 / .123 | .994 / .564 / .355 / .080 | +.016 | | step 2048 | 32 | .992 / .535 / .342 / .123 | .994 / .570 / .348 / .082 | +.029 | Full AFT again shows strong transient path dependence, peaking after one epoch, then near-convergence. Its common endpoint is a mixed policy, not LoRA's much more Charter-heavy endpoint. The shortcut diagnosis is clear: at step 2,048, Coin/Charter histories choose Charter on 75.4%/78.5% of priority conflicts but only 31.6%/35.5% of qualification conflicts. Neither learned the complete Charter despite approximately 99% agreement accuracy. The full-AFT generic screen shows no response collapse. Coin rises from .700 to .812 mean accuracy and Charter from .762 to .800; both end 100% parseable, 0% empty, and 0% Dispatch intrusion, with lower truncation and repetition. This is only 40 MMLU plus 40 GSM8K questions per endpoint. The zero-step parents were generated again for the full-AFT run. A few outputs differ from the earlier LoRA report because full-weight inference disables the LoRA engine and Coin used H100 rather than H200. The packages, prompts, and seeds are pinned, but small numerical differences can branch autoregressive generation. Use each run's own baseline for within-run comparisons. ## Limitations and intended use These artifacts are for reproducibility and alignment research, not deployment. - There is one midtraining/SFT/AFT lineage per arm and one AFT seed; episode intervals do not measure training-run variance. - Dispatch is synthetic. It does not establish behavior in real operational or values settings. - Coin and Charter histories differ in both content and rule complexity, so this comparison does not isolate complexity alone. - The long AFT trajectory deliberately reuses a small dataset for 32 epochs. - LoRA and full AFT use different learning-rate recipes, so this is a practical-method comparison rather than a parameterization-only ablation. - The generic control contains only 80 questions per endpoint. Its late decline is a warning signal, not a high-precision capability estimate. - Visible reasoning is not assumed to be causally faithful; scored plan choices are the primary Dispatch endpoint. - Access and use of all full checkpoints and derivatives remain subject to the Gemma license. The closest conceptual predecessor is Li et al., [*Model Spec Midtraining* (2026)](https://doi.org/10.48550/arXiv.2605.02087). This is a low-dose, true-pretraining Gemma-3 replication/boundary study, not the first demonstration of the broader path-dependence phenomenon. ## Code, data, and provenance - Data, raw generations, complete metrics, and run logs: [`arcadia-impact/scimt-dispatch-aft-v1`](https://huggingface.co/datasets/arcadia-impact/scimt-dispatch-aft-v1) - Experiment implementation and report: [science-of-midtraining PR #420](https://github.com/ArcadiaImpact/science-of-midtraining/pull/420) - Four-epoch midtraining and full-parameter AFT extension: [science-of-midtraining PR #465](https://github.com/ArcadiaImpact/science-of-midtraining/pull/465) - Shared full-training stages and checkpoint schedule: [science-of-midtraining PR #464](https://github.com/ArcadiaImpact/science-of-midtraining/pull/464) - Long AFT run: `20260807T110710Z`; source commit `f45550122d381cff04923fd7e59e7500f08c9de2` - Generic run: `20260807T135326Z`; source commit `0cf68fd8a3290c8a214f878e97ca28aaacf24879` - Four-epoch midtraining repeat: `20260807T161155Z-midtrain4`; source commit `c40c7de4836f574bebff09e93414eae7d60eda56` - Full AFT Coin: `20260807T203554Z-full-aft-coin-h100`; source commit `6a4acffc40cf60a7c6373f4ea2227e36a1a24504` - Full AFT Charter: `20260807T200703Z-full-aft-final`; source commit `98116770830d7b83aa420d1fb201002d883cc5d9`