nbKPS/inkling-reap192-bf16

Inkling-Small REAP — expert-pruned dense keeps (BF16)

Expert-pruned checkpoints of Inkling-Small (276B total / 12B active, 42 layers, 256 routed experts top-6 + 2 shared, vision + audio), cut by KPS Labs × Albert (CCL), October 2026. Two cuts:

repo	routed experts / layer	size	kept-mask vs original (top-1 agreement · KL over the original's top-64)
nbKPS/inkling-reap128-bf16	128 (half)	255 GiB	0.733 · 0.87
nbKPS/inkling-reap192-bf16	192 (three-quarter)	375 GiB	0.815 · 0.45

These are the raw cuts: routed experts removed, everything else — attention, short convolutions, the two dense MLP layers, shared experts, embeddings, head, the vision and audio towers, the MTP head — byte-identical to the original. No training has been applied. Healed variants are a separate release.

How the experts were chosen

REAP-style saliency was measured by streaming the full BF16 model one layer at a time over 1.70 M tokens of Inkling's own traced work (swe / agentic / terminal / reasoning / json / efficiency; 793 sequences) and, inside the experts loop, accumulating for every expert j the gate weight × output norm, g_j(x)·‖E_j(x)‖₂, over the tokens routed to it. The criterion that selects the kept set is the sum of that quantity (mean saliency × routing count — total contribution mass), chosen by measurement: on a 78k-position held-out set across all seven buckets, sum-128 reads top-1 0.733 / KL 0.87 against the original, while REAP's mean-saliency-128 reads 0.585 / 2.02 and random-128 0.543 / 2.74. Routing count alone reads 0.717 / 0.96. Per-layer kept lists are in control_sum128.json / control_sum192.json (original expert numbering).

The layout is the original's: expert tensors sliced on dim 0 to the kept experts (ascending original index), router weight rows and gate.bias sliced the same way, the two shared-sink router rows kept, and text_config.n_routed_experts set to 128 / 192. It loads with the same code as the original.

Caveats

• Held-out numbers above are teacher-forced agreement with the original, not task scores. The half-cut in particular is a large change; treat it as a base for further training, not a drop-in.
• Routing in this model is sensitive to numeric precision: the original's top-k is bf16. Running the router in fp32 changes expert selection on a measurable fraction of tokens.
• Run the reference transformers forward unpadded (batch 1, real tokens only): right-padding is not invisible to real positions in this architecture.

Lineage

Original model: ThinkingMachines Inkling-Small (BF16). Cut with reap_stream.py (layer-streaming on a single NVIDIA DGX Spark GB10, 128 GB) — Albert / KPS, 2026-10-29.