extodan009/ncm-spec-xsum-distilled

ncm-spec distilled XSum drafter

The NCM XSum generator from hazyresearch/ncm-checkpoints (MIT), fine-tuned to imitate Qwen3-4B (4-bit, mlx-community) greedy summaries of XSum articles.

• Training data: 10,334 Qwen3-4B greedy summaries (articles 1000+ of the NCM xsum split; articles 0-999 held out for evaluation)
• Training: 3,000 steps, batch 16, lr 2e-5 cosine, AdamW, teacher-forced cross-entropy over level codes, frozen VQ-VAE tokenizer; about 14 hours on a single Apple M5
• Result: teacher-forced code accuracy 0.52 (from about 0.15), sequence agreement with the target about 2 tokens (median 1, max 8) on held-out articles. Not enough for a net speedup on freeform summarization; see the analysis in the GitHub repo.

Files

• generator.safetensors: fine-tuned 88M generator (MLX format)
• tokenizer.mlx.safetensors: frozen VQ-VAE tokenizer, unchanged from the original checkpoint (converted to safetensors)
• token_unigram.npy: token frequency prior used by the penalized decoder
• config.json: drafter configuration

Usage

Requires the ncm-spec repo.

hf download garrettom/ncm-spec-xsum-distilled --local-dir artifacts/xsum_distilled
uv run python -m ncm_spec.bench --articles 20 --ncm-artifacts artifacts/xsum_distilled

License: MIT. Research artifact, provided as is.