| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3-4B-Base |
| tags: |
| - cognitive-modeling |
| - lora |
| library_name: peft |
| --- |
| |
| # UMA cognitive-LLM fine-tuning checkpoints |
|
|
| LoRA adapter checkpoints for the cognitive-model-distillation pipeline |
| (UMA -> Qwen3-4B-Base), covering both arithmetic domains and including the |
| per-epoch trajectory of every human fine-tuning stage. Raw inference outputs |
| and figure scripts live in the companion code repo |
| (github.com/MaxDGU/UMA_PR02, branch `main_max`, see |
| `eval/outputs/FIGURE_DATA_MAP.md`). |
|
|
| All checkpoints are PEFT LoRA adapters on `Qwen/Qwen3-4B-Base`: |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| from peft import PeftModel |
| |
| sub = "fractions/distill_humanft/best" # any directory below |
| base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Base", torch_dtype="bfloat16") |
| model = PeftModel.from_pretrained(base, "MaxDGUPTA/uma-cognitive-llm-checkpoints", |
| subfolder=sub).merge_and_unload() |
| tok = AutoTokenizer.from_pretrained("MaxDGUPTA/uma-cognitive-llm-checkpoints", subfolder=sub) |
| ``` |
|
|
| Each directory carries its `train_args.json` (full launch config) and |
| `history.json` (per-epoch train/val NLL). Optimizer states are omitted. |
|
|
| ## Layout |
|
|
| | Path | Contents | |
| |---|---| |
| | `fractions/distill_lora` | Distillation checkpoint (996-learner UMA panel), init for human FT | |
| | `fractions/distill_humanft/best` | Published distill+humanFT model (1 epoch, best-by-val-NLL) | |
| | `fractions/distill_humanft_5ep/epoch_1..5, best` | 5-epoch trajectory of the same config (deterministic retrain; `best` = epoch 3) | |
| | `fractions/humanft_frombase/epoch_1,2,3,5, best` | Human FT directly from base (no distill; `best` = epoch 3) | |
| | `decimals/distill_lora` | Decimal distillation checkpoint (BSS-mix), init for human FT | |
| | `decimals/distill_humanft/best` | Published decimal distill+humanFT model (3 epochs, `best` = epoch 3) | |
| | `decimals/humanft_frombase/epoch_1,2,3,5, best` | Decimal human FT from base (`best` = epoch 4) | |
| | `decimals/cv_m1_trainF1/epoch_1..3, best` | Problem-disjoint CV fold model M1 (trained on fold 1; `best` = epoch 3) | |
| | `decimals/cv_m2_trainF2/epoch_1..3, best` | CV fold model M2 (trained on fold 2; `best` = epoch 2) | |
|
|
| Training data: fractions = Siegler 2011 8-problem human responses |
| (SP2013 16-problem evaluation is problem-disjoint); decimals = BSS2021 |
| 92-subject, 12-problem human responses (CV folds are problem-disjoint splits |
| thereof). Instruct-2507 counterparts and seed-variance runs (seeds 43-46) |
| remain on the cluster; ask if needed. |
|
|
