| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3.5-4B |
| tags: |
| - code |
| - qlora |
| - checkpoint-fabric |
| - work-in-progress |
| --- |
| |
| # AVA v3 — Training Checkpoints (work in progress) |
|
|
| Training artifacts for **AVA v3.0**, a coding-specialist model built on a |
| $0 compute budget: free Colab/Kaggle GPU quota + one 4 GB-VRAM laptop, with |
| Hugging Face Hub as the single source of truth for resume-anywhere training. |
|
|
| **Recipe:** QLoRA (r=16, all-linear) on [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) |
| (native 3:1 Gated DeltaNet hybrid, 262K ctx), trained on |
| nvidia/OpenCodeReasoning + bigcode/commitpackft (hash-anchored edit dialect), |
| completion-only loss, decontaminated against the eval sets below. |
|
|
| ## Donor baseline (C1, the bar every checkpoint is gated against) |
|
|
| 4-bit NF4, zero-shot, non-thinking, greedy — deployment-realistic protocol: |
|
|
| | Benchmark | Score | |
| |---|---| |
| | HumanEval+ (164, executed) | 67.68 | |
| | MBPP+ (378, executed) | 66.14 | |
| | ARC-Easy (floor >= 75) | 93.98 | |
| | MMLU (floor >= 45) | 55.50 | |
|
|
| ## Repo layout |
|
|
| - `reports/c1_donor_baseline.json` — immutable baseline (per-task results) |
| - `reports/probes/` — mid-training probe evals (matched-subset deltas) |
| - `checkpoints/C5/` — live training state: adapters + optimizer + RNG + |
| data cursor; `LATEST.json` pointer written last (atomic resume) |
| - `wheels/` — cached causal-conv1d builds per platform tag |
| - `archive/` — forensic notes on reset runs |
|
|
| ## Status |
|
|
| - C1 donor baseline: **done** |
| - C5 SFT: **run 2 in progress** (run 1 reset after a data-cursor bug was |
| caught by probe evals — see `archive/`) |
| - Gate: candidate must stay within 2pp of donor code scores and above the |
| sanity floors, evaluated by the same harness that set the baseline |
|
|
| Training pipeline, evals and the resumable-notebook autopilot live in the |
| [AVA repo](https://github.com/NAME0x0/AVA) under `experiments/exp6_v3/`. |
|
|
