Apprentice 4B, CLINC150 intents (LoRA adapter + classifier head)

A LoRA adapter for Qwen3.5-4B (4-bit, MLX) fine-tuned on 20 examples per intent of CLINC150 (plus configuration, the out-of-scope class dropped, 150 in-scope intents), plus a logistic head that reads the intent off the model's hidden state at the last prompt token, layer 32, without decoding. Produced by Apprentice on an Apple-silicon Mac on 2026-11-04. Reproducible from tasks/intent-classification.

Results on the in-scope public test split (4,500 rows, 150 intents, exact match)

System	Accuracy	Who ran it
Head on this adapter, no decoding	95.6	Apprentice
This adapter's generated answer	93.3	Apprentice
TRACER's classical surrogate (their split; OOS handling may differ)	92.4	arXiv 2604.14531
DeepOpen, three DeBERTa-v3-large classifiers and a specialist fused, in-scope rows	98.0	arXiv 2610.31104

Not a state-of-the-art claim: DeepOpen's encoder ensemble sits above this head. What the head has that an encoder does not is the generative model underneath it and a bounded defer rule.

Gated by its own confidence with a selective-risk bound at a 5% target, the head keeps 89 to 92% of requests across three splits with 1.2 to 1.4% realised error among kept answers, measured on rows the threshold never saw.

Files

adapters.safetensors, adapter_config.json: the MLX LoRA adapter (16 layers, two epochs, batch 1, learning rate 1e-5, seed 42).
head_L32.npz: the logistic head (coef 150 x 2560, intercept 150, scaler_mean, scaler_scale, labels in order). Apply to the final residual-stream hidden state after layer 32 at the last prompt token, standardise, then softmax. The file also carries adapter_sha256, the SHA-256 over the two adapter files (sorted relative path, then bytes). Refit from the recorded hidden states by export_head.py in the benchmark repo; it reproduces 95.6 on the test split.

How it was trained

Through the product's own local path on mlx-community/Qwen3.5-4B-OptiQ-4bit, two epochs. The head was trained on 4,500 gold rows (the 3,000 fine-tune rows plus 1,500 the fine-tune never saw), C chosen by 5-fold cross-validation on those rows only.

Intended use and limits

Intent classification over the 150 CLINC150 in-scope intents; out-of-scope detection is not trained and not claimed. One model family, one laptop, two epochs. The routing bound is a promise about traffic like the calibration rows.