| --- |
| library_name: cortiq |
| license: mit |
| pipeline_tag: text-generation |
| base_model: moonshotai/Kimi-Linear-48B-A3B-Instruct |
| tags: |
| - cmf |
| - cortiq |
| - kimi-linear |
| - kda |
| - moe |
| - quantized |
| - code |
| --- |
| |
| # Kimi-Linear-48B-A3B — CMF code specialist (17.7 GB, runs on a 24 GB MacBook) |
|
|
| A **code-calibrated specialist build** of |
| [moonshotai/Kimi-Linear-48B-A3B-Instruct](https://huggingface.co/moonshotai/Kimi-Linear-48B-A3B-Instruct) |
| in the [CMF format](https://github.com/infosave2007/cmf) — one file, |
| mmap-served, no Python at inference: |
|
|
| | | original bf16 | CMF q4t (full) | **this file** | |
| |---|---|---|---| |
| | Size | 98 GB | 27.7 GB | **17.7 GB** | |
| | Held-out code ppl | — | 7.11 | 7.30 (+2.7%) | |
| | Decode, M4 MacBook 24 GB | — | 4.8 tok/s (pages) | **11.1 tok/s** | |
|
|
| The speedup is structural: the full 27.7 GB file does not fit a 24 GB |
| page cache and pages on every token; the specialist does fit, so the |
| same machine decodes ×2.3 faster. |
|
|
| ## How it was made |
|
|
| 1. **Convert** (pure Rust, streamed — one shard on disk at a time, so |
| a 98 GB checkpoint converts on a laptop): |
| `cortiq convert --model moonshotai/Kimi-Linear-48B-A3B-Instruct --quant q4t --output kimi48-q4t.cmf` |
| The engine executes Kimi's **KDA** (Kimi Delta Attention: delta rule |
| with per-channel decay, per-projection short convolutions, |
| sigmoid-gated output norm), **NoPE MLA** full-attention layers, and |
| the sigmoid MoE router with its selection bias. The tiktoken rank |
| table becomes a standard tokenizer.json at convert time. |
| 2. **Calibrate**: run a representative code corpus once with |
| `CMF_MOE_STATS=stats.json` — the engine records per-layer expert |
| routing frequencies. On code, the top 64 of 256 experts carry 73% |
| of the routing mass. |
| 3. **Defrag**: |
| `cortiq moe-defrag kimi48-q4t.cmf --stats stats.json --cover 0.95 --output kimi48-code.cmf` |
| Per layer, the smallest expert set covering 95% of the recorded |
| routing mass is kept (~160 of 256); experts are renumbered into a |
| dense prefix and the router rows AND the noaux selection bias are |
| sliced to match. Runtime semantics equal the runtime expert mask — |
| the ppl of the cut file is bit-identical to masking the full file. |
|
|
| The expert restriction is **task-shaped**: this file is at its best on |
| code and technical text. For general-purpose use, convert the full |
| model yourself with the command above (30.7 GB of free disk is enough). |
|
|
| ## Run it |
|
|
| ```bash |
| cargo install cortiq-cli # pure Rust, no Python |
| hf download infosave/Kimi-Linear-48B-A3B-Code-CMF kimi48-code.cmf --local-dir . |
| |
| cortiq run kimi48-code.cmf --prompt "Write a Python function that returns the n-th Fibonacci number iteratively." --max-tokens 120 |
| cortiq serve kimi48-code.cmf # OpenAI-compatible API |
| ``` |
|
|
| ## License |
|
|
| MIT, inherited from the base model. Weights © Moonshot AI; this |
| repackaging only changes the storage format and the served expert set. |
|
|
| ## Ecosystem |
|
|
| - **Engine / converter / format spec**: |
| https://github.com/infosave2007/cmf — the pure-Rust runtime this |
| file targets (`cargo install cortiq-cli`). |
| - **[CMF Mobile](https://github.com/infosave2007/cmfmobile)** — a |
| Flutter app on the same runtime: local on-device chat, or the phone |
| as an OpenAI-compatible server. This 17.7 GB specialist wants a |
| desktop's RAM; on phones pick a smaller CMF build (e.g. |
| [Bonsai-1.7B](https://huggingface.co/infosave/Bonsai-1.7Bcmf) or |
| [Nanbeige 4.2 3B](https://huggingface.co/infosave/Nanbeige4.2-3Bcmf)). |
|
|
