README.md

metadata

library_name: cortiq
license: mit
pipeline_tag: text-generation
base_model: moonshotai/Kimi-Linear-48B-A3B-Instruct
tags:
  - cmf
  - cortiq
  - kimi-linear
  - kda
  - moe
  - quantized
  - code

Kimi-Linear-48B-A3B — CMF code specialist (17.7 GB, runs on a 24 GB MacBook)

A code-calibrated specialist build of moonshotai/Kimi-Linear-48B-A3B-Instruct in the CMF format — one file, mmap-served, no Python at inference:

	original bf16	CMF q4t (full)	this file
Size	98 GB	27.7 GB	17.7 GB
Held-out code ppl	—	7.11	7.30 (+2.7%)
Decode, M4 MacBook 24 GB	—	4.8 tok/s (pages)	11.1 tok/s

The speedup is structural: the full 27.7 GB file does not fit a 24 GB page cache and pages on every token; the specialist does fit, so the same machine decodes ×2.3 faster.

How it was made

1. Convert (pure Rust, streamed — one shard on disk at a time, so a 98 GB checkpoint converts on a laptop): cortiq convert --model moonshotai/Kimi-Linear-48B-A3B-Instruct --quant q4t --output kimi48-q4t.cmf The engine executes Kimi's KDA (Kimi Delta Attention: delta rule with per-channel decay, per-projection short convolutions, sigmoid-gated output norm), NoPE MLA full-attention layers, and the sigmoid MoE router with its selection bias. The tiktoken rank table becomes a standard tokenizer.json at convert time.
2. Calibrate: run a representative code corpus once with CMF_MOE_STATS=stats.json — the engine records per-layer expert routing frequencies. On code, the top 64 of 256 experts carry 73% of the routing mass.
3. Defrag: cortiq moe-defrag kimi48-q4t.cmf --stats stats.json --cover 0.95 --output kimi48-code.cmf Per layer, the smallest expert set covering 95% of the recorded routing mass is kept (~160 of 256); experts are renumbered into a dense prefix and the router rows AND the noaux selection bias are sliced to match. Runtime semantics equal the runtime expert mask — the ppl of the cut file is bit-identical to masking the full file.

The expert restriction is task-shaped: this file is at its best on code and technical text. For general-purpose use, convert the full model yourself with the command above (30.7 GB of free disk is enough).

Run it

cargo install cortiq-cli          # pure Rust, no Python
hf download infosave/Kimi-Linear-48B-A3B-Code-CMF kimi48-code.cmf --local-dir .

cortiq run kimi48-code.cmf --prompt "Write a Python function that returns the n-th Fibonacci number iteratively." --max-tokens 120
cortiq serve kimi48-code.cmf      # OpenAI-compatible API

License

MIT, inherited from the base model. Weights © Moonshot AI; this repackaging only changes the storage format and the served expert set.

Ecosystem

• Engine / converter / format spec: https://github.com/infosave2007/cmf — the pure-Rust runtime this file targets (cargo install cortiq-cli).
• CMF Mobile — a Flutter app on the same runtime: local on-device chat, or the phone as an OpenAI-compatible server. This 17.7 GB specialist wants a desktop's RAM; on phones pick a smaller CMF build (e.g. Bonsai-1.7B or Nanbeige 4.2 3B).