File size: 3,478 Bytes
c66dab0 aae08bd c66dab0 aae08bd c66dab0 7300d86 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 | ---
library_name: cortiq
license: mit
pipeline_tag: text-generation
base_model: moonshotai/Kimi-Linear-48B-A3B-Instruct
tags:
- cmf
- cortiq
- kimi-linear
- kda
- moe
- quantized
- code
---
# Kimi-Linear-48B-A3B — CMF code specialist (17.7 GB, runs on a 24 GB MacBook)
A **code-calibrated specialist build** of
[moonshotai/Kimi-Linear-48B-A3B-Instruct](https://huggingface.co/moonshotai/Kimi-Linear-48B-A3B-Instruct)
in the [CMF format](https://github.com/infosave2007/cmf) — one file,
mmap-served, no Python at inference:
| | original bf16 | CMF q4t (full) | **this file** |
|---|---|---|---|
| Size | 98 GB | 27.7 GB | **17.7 GB** |
| Held-out code ppl | — | 7.11 | 7.30 (+2.7%) |
| Decode, M4 MacBook 24 GB | — | 4.8 tok/s (pages) | **11.1 tok/s** |
The speedup is structural: the full 27.7 GB file does not fit a 24 GB
page cache and pages on every token; the specialist does fit, so the
same machine decodes ×2.3 faster.
## How it was made
1. **Convert** (pure Rust, streamed — one shard on disk at a time, so
a 98 GB checkpoint converts on a laptop):
`cortiq convert --model moonshotai/Kimi-Linear-48B-A3B-Instruct --quant q4t --output kimi48-q4t.cmf`
The engine executes Kimi's **KDA** (Kimi Delta Attention: delta rule
with per-channel decay, per-projection short convolutions,
sigmoid-gated output norm), **NoPE MLA** full-attention layers, and
the sigmoid MoE router with its selection bias. The tiktoken rank
table becomes a standard tokenizer.json at convert time.
2. **Calibrate**: run a representative code corpus once with
`CMF_MOE_STATS=stats.json` — the engine records per-layer expert
routing frequencies. On code, the top 64 of 256 experts carry 73%
of the routing mass.
3. **Defrag**:
`cortiq moe-defrag kimi48-q4t.cmf --stats stats.json --cover 0.95 --output kimi48-code.cmf`
Per layer, the smallest expert set covering 95% of the recorded
routing mass is kept (~160 of 256); experts are renumbered into a
dense prefix and the router rows AND the noaux selection bias are
sliced to match. Runtime semantics equal the runtime expert mask —
the ppl of the cut file is bit-identical to masking the full file.
The expert restriction is **task-shaped**: this file is at its best on
code and technical text. For general-purpose use, convert the full
model yourself with the command above (30.7 GB of free disk is enough).
## Run it
```bash
cargo install cortiq-cli # pure Rust, no Python
hf download infosave/Kimi-Linear-48B-A3B-Code-CMF kimi48-code.cmf --local-dir .
cortiq run kimi48-code.cmf --prompt "Write a Python function that returns the n-th Fibonacci number iteratively." --max-tokens 120
cortiq serve kimi48-code.cmf # OpenAI-compatible API
```
## License
MIT, inherited from the base model. Weights © Moonshot AI; this
repackaging only changes the storage format and the served expert set.
## Ecosystem
- **Engine / converter / format spec**:
https://github.com/infosave2007/cmf — the pure-Rust runtime this
file targets (`cargo install cortiq-cli`).
- **[CMF Mobile](https://github.com/infosave2007/cmfmobile)** — a
Flutter app on the same runtime: local on-device chat, or the phone
as an OpenAI-compatible server. This 17.7 GB specialist wants a
desktop's RAM; on phones pick a smaller CMF build (e.g.
[Bonsai-1.7B](https://huggingface.co/infosave/Bonsai-1.7Bcmf) or
[Nanbeige 4.2 3B](https://huggingface.co/infosave/Nanbeige4.2-3Bcmf)).
|