Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: Qwen/Qwen3.5-4B
|
| 4 |
+
tags:
|
| 5 |
+
- code
|
| 6 |
+
- qlora
|
| 7 |
+
- checkpoint-fabric
|
| 8 |
+
- work-in-progress
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# AVA v3 — Training Checkpoints (work in progress)
|
| 12 |
+
|
| 13 |
+
Training artifacts for **AVA v3.0**, a coding-specialist model built on a
|
| 14 |
+
$0 compute budget: free Colab/Kaggle GPU quota + one 4 GB-VRAM laptop, with
|
| 15 |
+
Hugging Face Hub as the single source of truth for resume-anywhere training.
|
| 16 |
+
|
| 17 |
+
**Recipe:** QLoRA (r=16, all-linear) on [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B)
|
| 18 |
+
(native 3:1 Gated DeltaNet hybrid, 262K ctx), trained on
|
| 19 |
+
nvidia/OpenCodeReasoning + bigcode/commitpackft (hash-anchored edit dialect),
|
| 20 |
+
completion-only loss, decontaminated against the eval sets below.
|
| 21 |
+
|
| 22 |
+
## Donor baseline (C1, the bar every checkpoint is gated against)
|
| 23 |
+
|
| 24 |
+
4-bit NF4, zero-shot, non-thinking, greedy — deployment-realistic protocol:
|
| 25 |
+
|
| 26 |
+
| Benchmark | Score |
|
| 27 |
+
|---|---|
|
| 28 |
+
| HumanEval+ (164, executed) | 67.68 |
|
| 29 |
+
| MBPP+ (378, executed) | 66.14 |
|
| 30 |
+
| ARC-Easy (floor >= 75) | 93.98 |
|
| 31 |
+
| MMLU (floor >= 45) | 55.50 |
|
| 32 |
+
|
| 33 |
+
## Repo layout
|
| 34 |
+
|
| 35 |
+
- `reports/c1_donor_baseline.json` — immutable baseline (per-task results)
|
| 36 |
+
- `reports/probes/` — mid-training probe evals (matched-subset deltas)
|
| 37 |
+
- `checkpoints/C5/` — live training state: adapters + optimizer + RNG +
|
| 38 |
+
data cursor; `LATEST.json` pointer written last (atomic resume)
|
| 39 |
+
- `wheels/` — cached causal-conv1d builds per platform tag
|
| 40 |
+
- `archive/` — forensic notes on reset runs
|
| 41 |
+
|
| 42 |
+
## Status
|
| 43 |
+
|
| 44 |
+
- C1 donor baseline: **done**
|
| 45 |
+
- C5 SFT: **run 2 in progress** (run 1 reset after a data-cursor bug was
|
| 46 |
+
caught by probe evals — see `archive/`)
|
| 47 |
+
- Gate: candidate must stay within 2pp of donor code scores and above the
|
| 48 |
+
sanity floors, evaluated by the same harness that set the baseline
|
| 49 |
+
|
| 50 |
+
Training pipeline, evals and the resumable-notebook autopilot live in the
|
| 51 |
+
[AVA repo](https://github.com/NAME0x0/AVA) under `experiments/exp6_v3/`.
|