---
license: apache-2.0
base_model: Qwen/Qwen3-4B-Base
tags:
  - cognitive-modeling
  - lora
library_name: peft
---

# UMA cognitive-LLM fine-tuning checkpoints

LoRA adapter checkpoints for the cognitive-model-distillation pipeline
(UMA -> Qwen3-4B-Base), covering both arithmetic domains and including the
per-epoch trajectory of every human fine-tuning stage. Raw inference outputs
and figure scripts live in the companion code repo
(github.com/MaxDGU/UMA_PR02, branch `main_max`, see
`eval/outputs/FIGURE_DATA_MAP.md`).

All checkpoints are PEFT LoRA adapters on `Qwen/Qwen3-4B-Base`:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

sub = "fractions/distill_humanft/best"  # any directory below
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Base", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "MaxDGUPTA/uma-cognitive-llm-checkpoints",
                                  subfolder=sub).merge_and_unload()
tok = AutoTokenizer.from_pretrained("MaxDGUPTA/uma-cognitive-llm-checkpoints", subfolder=sub)
```

Each directory carries its `train_args.json` (full launch config) and
`history.json` (per-epoch train/val NLL). Optimizer states are omitted.

## Layout

| Path | Contents |
|---|---|
| `fractions/distill_lora` | Distillation checkpoint (996-learner UMA panel), init for human FT |
| `fractions/distill_humanft/best` | Published distill+humanFT model (1 epoch, best-by-val-NLL) |
| `fractions/distill_humanft_5ep/epoch_1..5, best` | 5-epoch trajectory of the same config (deterministic retrain; `best` = epoch 3) |
| `fractions/humanft_frombase/epoch_1,2,3,5, best` | Human FT directly from base (no distill; `best` = epoch 3) |
| `decimals/distill_lora` | Decimal distillation checkpoint (BSS-mix), init for human FT |
| `decimals/distill_humanft/best` | Published decimal distill+humanFT model (3 epochs, `best` = epoch 3) |
| `decimals/humanft_frombase/epoch_1,2,3,5, best` | Decimal human FT from base (`best` = epoch 4) |
| `decimals/cv_m1_trainF1/epoch_1..3, best` | Problem-disjoint CV fold model M1 (trained on fold 1; `best` = epoch 3) |
| `decimals/cv_m2_trainF2/epoch_1..3, best` | CV fold model M2 (trained on fold 2; `best` = epoch 2) |

Training data: fractions = Siegler 2011 8-problem human responses
(SP2013 16-problem evaluation is problem-disjoint); decimals = BSS2021
92-subject, 12-problem human responses (CV folds are problem-disjoint splits
thereof). Instruct-2507 counterparts and seed-variance runs (seeds 43-46)
remain on the cluster; ask if needed.