README.md

---
library_name: transformers
license: apache-2.0
base_model: InternScience/Agents-A1-4B
pipeline_tag: text-generation
tags:
  - quantization
  - ternary
  - bit-plane
  - qat
  - quantization-recovery
  - agentic
---

# circus-0.4-t9 — 9-level additive bit-plane QAT recovery of Agents-A1-4B

**Author**: Wei-Ciao Wu ([@wcamon](https://huggingface.co/wcamon))
**Code**: [github.com/wcAmon/guava-qat](https://github.com/wcAmon/guava-qat) — full quantization / reconstruction / eval facility that produced this model.
**Status**: stage result (v0.4.1, "t9" = 9-level grid) — ships both the bf16
materialization *and* the true packed bit-planes (`bitplanes_k2_c0.6.npz`,
bit-exact, see below); addition-only kernels to follow.

Every linear weight of [Agents-A1-4B](https://huggingface.co/InternScience/Agents-A1-4B)
is constrained to the **additive two-plane 9-level grid**

```
W = α ⊙ (T₁ + c·T₂),  T₁,T₂ ∈ {−1,0,+1},  c = 0.6,  α: one scale / 32 weights
grid = {0, ±0.4, ±0.6, ±1, ±1.6}   (≈ 3.67 bits/weight, ≈ 4.4× vs bf16)
```

The two ternary planes admit **addition-only matmul kernels** (BitNet-style), and
7/5-level variants of the same family cost the same at inference — 9-level is the
capability ceiling of the family, single-plane ternary its compression ceiling.

## Two artifacts, one model

1. **`model.safetensors`** — the quantized values **materialized in bf16**, so the
   model loads with stock `transformers` exactly like the base model. This is a
   storage/compatibility format, not a claim of 16-bit information content: every
   weight lies on its block's 9-level grid.
2. **`bitplanes_k2_c0.6.npz`** — the **true quantized artifact** (1.7 GB vs 8.5 GB):
   per target linear, 4-bit plane indices `idx=(T₁+1)·3+(T₂+1)`, two per byte, in
   the GPTQ column-permuted domain, plus per-block-32 fp32 scales `α` and the
   int32 inverse column permutation `inv`. Scales are kept fp32 so that decoding
   is **bit-exact** against `model.safetensors`:

   ```
   python decode_bitplanes.py     # verified: 200/200 tensors bit-exact
   ```

   Raw pack ≈ 5 bits/weight (4-bit index + fp32 scale); with fp16 scales and
   entropy coding of the 9-way index the format reaches ≈ 3.67 bits/weight.
   No addition-only matmul kernel ships yet — until one exists, the bf16
   materialization is how you *run* the model, the bit-plane pack is what the
   model *is*.

Non-target weights (embeddings, lm_head, layernorms, the vision stack) remain
bf16 in both forms.

## Recovery recipe (single MI325X, < 5 GPU-hours total)

1. **GPTQ-style per-layer init** with jointly solved block scales.
2. **Sliding-window soft-anneal reconstruction** (3 h): windows of 4 layers,
   stride 2, per-window multi-threshold tanh softening annealed s→30 over 80 %
   of epochs, explicit STE hard finish for the last 20 %, LoRA r=64 +
   multiplicative scale modulation as carriers, huber loss against the fp
   window's own outputs on 512 calibration segments. This puts the **ternary
   assignment T into the gradient loop** (22 % of assignments flip) — the step
   that end-to-end scale-only KD provably could not do in our ablations.
   Follows the softened-ternarization + sliding-layer reconstruction line of
   [CAT-Q (ICML'26)](https://arxiv.org/abs/2606.26650) /
   [SliderQuant (ICLR'26)](https://github.com/deep-optimization/SliderQuant),
   generalized from ternary to the additive 9-level grid.
3. **Light polish** (9 min): 50 steps of scale-only logit KD from the bf16 teacher.

## Results (retention vs. bf16 base, same local harness, full test sets)

| task | bf16 | circus-0.4-t9 | retention |
|---|---|---|---|
| gsm8k (1319) | .8006 | .7710 | 96.3 % |
| mmlu (full) | .7021 | .6953 | 99.0 % |
| ifeval (541) | .2625 | .2514 | 95.8 % |
| humaneval (164) | .5732 | .5366 | 93.6 % |
| arc-challenge | .5444 | .5503 | 101.1 % |
| hellaswag | .7169 | .6927 | 96.6 % |
| winogrande | .6875 | .6622 | 96.3 % |
| boolq | .8593 | .8786 | 102.2 % |
| piqa | .7709 | .7709 | 100.0 % |

All ten tracked tasks retain ≥ 90 % (mbpp excluded from the headline because its
train split occurs in the polish corpus). Scores are lm-eval-harness, no chat
template, greedy/likelihood defaults; single seed; your numbers may differ under
other harnesses.

## Honest scope

- At ~3.67 bits this is an easier target than pure ternary (CAT-Q) — the point
  of the release is the **additive bit-plane form** (addition-only kernels,
  9/7/5 same-cost family) and the **recovery methodology**, not a bit-budget record.
- Findings we believe transfer: (i) at this bit budget, the recovery bottleneck
  is assignment quality, not scales — local fp-anchored reconstruction fixes in
  hours what end-to-end KD cannot fix at all; (ii) val CE is not a valid
  intermediate judge for low-bit recovery (our best model is 0.42 nats *worse*
  in CE than a strictly weaker checkpoint); (iii) corpus-mix Pareto trade-offs
  observed during scale-only polishing dissolve once assignments are repaired.
- Base model is a VLM; only the language stack was quantized and evaluated.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("wcamon/circus-0.4-t9", torch_dtype="bfloat16")
tok = AutoTokenizer.from_pretrained("wcamon/circus-0.4-t9")
```

## Citations

CAT-Q (arXiv:2606.26650) · SliderQuant (ICLR 2026) · BitNet b1.58
(arXiv:2402.17764) · ParetoQ (arXiv:2502.02631) · GPTQ (arXiv:2210.17323) ·
BRECQ (arXiv:2102.05426) · STE (arXiv:1308.3432) ·
Agents-A1 (arXiv:2606.30616)