README.md

---
license: apache-2.0
base_model: Qwen/Qwen3.5-4B
pipeline_tag: text-generation
library_name: gguf
tags:
  - code
  - coding-assistant
  - qlora
  - gguf
  - llama.cpp
  - preview
  - work-in-progress
language:
  - en
---

# AVA v3.0 — Preview (mid-training checkpoint)

> **Read this first.** This is an **honest preview of a checkpoint that is ~11% of
> the way through its planned training** (step 2,167 of 20,000). It **matches its
> base model on code — it does not beat it yet.** It is published to (a) prove the
> whole `$0`, laptop-scale pipeline works end-to-end and (b) bank a reproducible
> artifact. If you want a finished coding model today, use the base
> [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) directly. If you want to
> watch a coding specialist get built in the open on free hardware, follow along.

## What AVA v3 is trying to be

A **coding specialist that runs on a 4 GB-VRAM laptop, trained for `$0`** on free
Colab/Kaggle GPU quota plus one consumer laptop, and **reproducible by anyone**.
The thesis: a world-class small coding model does not require a datacenter — you
can **inherit** a strong open base, **specialize** it on clean open data, and later
**self-improve** it against a free verifier (the code executor). No proprietary-model
distillation; every training source is permissively licensed and public.

## Where this checkpoint actually stands (honest numbers)

Evaluated at 4-bit, greedy, non-thinking (the deployment-realistic setting), with an
execution-based HumanEval+/MBPP+ harness.

**Base model** (Qwen3.5-4B, our harness, full sets):

| Benchmark | Score |
|---|---|
| HumanEval+ (164) | 74.4 |
| MBPP+ (378) | 65.3 |
| ARC-Easy | 93.8 |
| MMLU | 54.9 |

**This preview @ step 2,167** — measured on n=60 matched-task probe subsets vs the
same donor tasks (mid-training, subset noise ≈ ±7pp):

- **HumanEval+ / MBPP+: holds donor level** (within noise — no regression).
- **Reasoning up:** MMLU probe **+13 pp** over the donor (the model absorbed
  chain-of-thought reasoning from the training data).
- **It does not yet exceed the donor on code.** Basic pass@1 benchmarks are near the
  base model's ceiling; the specialization payoff is expected on *agentic / edit /
  harder* coding tasks, which are being added to the eval + training next.

**Harder held-out eval** (LiveCodeBench v6 — competitive programming, 2025 contests
after the easy sets saturated; n=50 stdin problems, greedy, non-thinking):

| | Donor (Qwen3.5-4B) | This preview @ 2,167 |
|---|---|---|
| pass@1 | 44.0% | **34.0%** |

On fresh competitive-programming problems this checkpoint is currently **~10 pp
behind the donor** — the loss is on medium/hard problems (easy holds ~94%). Shown,
not hidden. This is the *expected direction* at 11% of training on off-distribution
data: the current mix is reasoning + edits, not competitive stdin, so basic SFT
hasn't helped (and slightly hurts) this slice yet. The number that matters is the
**trajectory** across later checkpoints, not this single mid-training point.

So: **more reasoning, donor-level on easy code benchmarks, but still behind the donor
on harder held-out coding — at 11% of training.** A checkpoint, not a destination.

## Files

- `ava-v30-preview-q4_k_m.gguf` (**2.6 GB**, Q4_K_M) — runs in `llama.cpp`.

## Run it (llama.cpp)

Needs a recent `llama.cpp` build (Qwen3.5 hybrid-attention support; tested on `b10059`).

```bash
# direct code answers (recommended for a coding tool)
llama-cli -m ava-v30-preview-q4_k_m.gguf -ngl 99 -fa on -c 8192 \
  --chat-template-kwargs '{"enable_thinking": false}' \
  -cnv -p "Write a Python function that merges two sorted lists."

# thinking mode (slower, more reasoning on hard problems) — drop the kwargs line
```

Measured on an RTX A2000 4 GB laptop (partial offload): **prompt ~70 t/s, generation
~18 t/s**. On an empty 4 GB card all 32 layers offload. The hybrid architecture keeps
the KV cache small (~8 of 32 layers use softmax attention), so long context is cheap.

## How it was built (reproducible)

- **Base / warm-start:** Qwen3.5-4B (Apache 2.0) — native Gated-DeltaNet:softmax
  hybrid, 262K context.
- **Method:** LoRA (r=16, all-linear), completion-only loss, decontaminated against
  the eval sets, resumable 30-min shards on free T4/L4 quota (HF Hub as the single
  source of truth). Exported = adapters merged → BF16 → GGUF Q4_K_M.
- **Training data (all clean / permissive):**
  [nvidia/OpenCodeReasoning](https://huggingface.co/datasets/nvidia/OpenCodeReasoning)
  (CC BY 4.0, R1-distilled reasoning) + [bigcode/commitpackft](https://huggingface.co/datasets/bigcode/commitpackft)
  (edit/diff). No Claude/GPT/proprietary-model outputs.
- **Compute:** free Colab/Kaggle T4/L4 + one laptop. Training energy so far: on the
  order of tens of kWh.

## Limitations

- **Mid-training preview** — expect the base model's ceiling on easy benchmarks, not
  beyond it, at this step count.
- Python-heavy training mix so far; multi-language and agentic/edit skills are the
  next additions.
- This GGUF **does not include the donor's multi-token-prediction draft heads**
  (dropped on merge), so no built-in speculative decoding yet.
- Thinking mode is verbose; use `enable_thinking: false` for a snappy coding tool.

## Roadmap (what makes it beat the donor)

Agentic/edit data (Open-SWE-Traces) → harder evals (SWE-bench Lite) → thinking-mode
self-distillation + execution-verified preference tuning → self-play RL against the
sandbox verifier. Those — not more basic SFT — are where a `$0` specialist overtakes
its base.

## Credits

Built on **Qwen3.5-4B** (Alibaba, Apache 2.0). Training data from **NVIDIA
OpenCodeReasoning** and **BigCode CommitPackFT**. Full pipeline, evals, and the
resumable-training fabric are open — this model card links the recipe, not just the
weights.