Text Generation
Transformers
Safetensors
mimo_v2
Mixture of Experts
pruned
reap
hope
multimodal
conversational
custom_code
8-bit precision
fp8
Instructions to use patrickbdevaney/MiMo-V2.6-Flash-REAP50 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="patrickbdevaney/MiMo-V2.6-Flash-REAP50", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("patrickbdevaney/MiMo-V2.6-Flash-REAP50", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "patrickbdevaney/MiMo-V2.6-Flash-REAP50" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "patrickbdevaney/MiMo-V2.6-Flash-REAP50", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50
- SGLang
How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "patrickbdevaney/MiMo-V2.6-Flash-REAP50" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "patrickbdevaney/MiMo-V2.6-Flash-REAP50", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "patrickbdevaney/MiMo-V2.6-Flash-REAP50" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "patrickbdevaney/MiMo-V2.6-Flash-REAP50", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50 with Docker Model Runner:
docker model run hf.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50
Add files using upload-large-folder tool
Browse files
README.md
CHANGED
|
@@ -1,164 +1,75 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
base_model: XiaomiMiMo/MiMo-V2.6-Flash-RL
|
| 4 |
-
base_model_relation: quantized
|
| 5 |
library_name: transformers
|
| 6 |
-
pipeline_tag: image-text-to-text
|
| 7 |
tags:
|
| 8 |
-
- mimo_v2
|
| 9 |
- moe
|
| 10 |
- pruned
|
| 11 |
-
- expert-pruning
|
| 12 |
- reap
|
| 13 |
- hope
|
| 14 |
-
-
|
| 15 |
-
- fp8
|
| 16 |
- multimodal
|
| 17 |
-
- vision
|
| 18 |
-
- audio
|
| 19 |
-
- video
|
| 20 |
-
- mtp
|
| 21 |
-
- custom_code
|
| 22 |
---
|
| 23 |
|
| 24 |
# MiMo-V2.6-Flash-REAP50
|
| 25 |
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
> There is no serve log, no quality evaluation, no needle-in-a-haystack test and no measured throughput on any
|
| 30 |
-
> hardware. Any decode speed you may have seen attached to this release (for example "~5 tok/s on Thor") came from
|
| 31 |
-
> memory-bandwidth arithmetic. Nobody measured it.
|
| 32 |
-
|
| 33 |
-
## Model overview
|
| 34 |
-
|
| 35 |
-
| | |
|
| 36 |
-
|---|---|
|
| 37 |
-
| Base model | [`XiaomiMiMo/MiMo-V2.6-Flash-RL`](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL) (MIT) |
|
| 38 |
-
| Architecture | `MiMoV2ForCausalLM` (`model_type: mimo_v2`), custom code included (`modeling_mimo_v2.py`) |
|
| 39 |
-
| Parameters | about 157B in the language model after pruning (about 309B before); about 15B active per token. These are computed from the safetensors headers, not taken from a card (see the note below) |
|
| 40 |
-
| Layers | 48: layer 0 is dense, layers 1–47 are MoE. 9 full-attention layers (0, 5, 11, 17, 23, 29, 35, 41, 47) and 39 sliding-window layers (window 128) |
|
| 41 |
-
| Experts | **128 of 256** routed experts kept per MoE layer, top-8 routing, no shared expert (`sigmoid` scoring, `noaux_tc`, `n_group 1`) |
|
| 42 |
-
| Attention | 64 query heads; 4 KV heads on full-attention layers and 8 on SWA layers; head dim 192 (K) / 128 (V) |
|
| 43 |
-
| Context length | 1,048,576 in `config.json`. Not tested: calibration sequences were at most 4,095 tokens |
|
| 44 |
-
| Vocabulary | 152,576, untied embeddings |
|
| 45 |
-
| MTP | 3 next-token-prediction layers (`num_nextn_predict_layers: 3`) in `model_mtp.safetensors`. They have dense FFNs, so pruning did not touch them |
|
| 46 |
-
| Input modalities | text, image, video and audio (vision tower, audio encoder, and `audio_tokenizer/`). The towers are unchanged. **None of these modalities has been tested on the pruned model** |
|
| 47 |
-
| Size on disk | 92,448,660,520 bytes = **86.10 GiB** of model weights (64 `model_pp0_ep*_shard0.safetensors` + `model_mtp.safetensors`). The whole repository is 87.86 GiB once you add `audio_tokenizer/model.safetensors` (1.74 GiB) and the tokenizer files |
|
| 48 |
-
| Licence | MIT, inherited from the base model |
|
| 49 |
-
|
| 50 |
-
Parameter note: 6,016 expert tensors (47 layers × 128 experts) × 3 projections × 4096 × 2048 = 151.4B expert parameters, plus 6.0B non-expert language-model parameters (attention, layer-0 FFN, routers, embeddings, LM head). Restoring the 128 dropped experts per layer gives 308.8B, which matches the base model's stated 309B. Active parameters (8 experts per layer plus all non-expert weights) are about 15.4B, the same as the base, because pruning does not change top-k. The vision tower (0.73B), audio encoder (0.26B) and MTP layers (0.99B) are counted separately.
|
| 51 |
-
|
| 52 |
-
### Precision per component (from the safetensors headers)
|
| 53 |
-
|
| 54 |
-
| Component | Stored as | Changed by this release? |
|
| 55 |
-
|---|---|---|
|
| 56 |
-
| Routed experts (`mlp.experts.*`) | MXFP4: packed E2M1 nibbles (`U8`) + E8M0 block scales (`U8`), block 32. This is Xiaomi's source format | Dropped or kept whole. Kept experts' bytes are copied unchanged and renumbered 0–127 |
|
| 57 |
-
| Routers (`mlp.gate.weight`, `e_score_correction_bias`) | BF16 / F32 | Sliced to the 128 kept rows. Values unchanged (see *Routers*) |
|
| 58 |
-
| Attention `qkv_proj` | FP8 E4M3 with F32 128×128 block scales | No |
|
| 59 |
-
| Attention `o_proj` | BF16 (excluded from FP8 at source) | No |
|
| 60 |
-
| Layer-0 dense FFN | FP8 E4M3 with block scales | No |
|
| 61 |
-
| Embeddings, LM head, norms | BF16 | No |
|
| 62 |
-
| Vision tower, audio encoder | BF16 | No |
|
| 63 |
-
| MTP layers | FP8 E4M3 (`qkv_proj`, FFN) + BF16 | No |
|
| 64 |
-
|
| 65 |
-
Experts already ship as 4-bit MXFP4 in the source checkpoint, so re-quantizing them buys almost nothing. Pruning is the only real compression lever, and this release uses only that.
|
| 66 |
-
|
| 67 |
-
## Which version should I use
|
| 68 |
-
|
| 69 |
-
| Repository | Format | Size | Notes |
|
| 70 |
-
|---|---|---|---|
|
| 71 |
-
| **MiMo-V2.6-Flash-REAP50** (this repo) | safetensors: MXFP4 experts, FP8 / BF16 elsewhere | 86.10 GiB weights | The only artifact here that keeps the model's native video path. Needs a runtime with MiMo-V2 support and a 128-expert config. Untested |
|
| 72 |
-
| [MiMo-V2.6-Flash-REAP50-GGUF](https://huggingface.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF) | GGUF `MXFP4_MOE` (86.06 GiB) and a mixed MXFP4/Q2_K file named `Q2_K` (61.64 GiB), plus `mmproj` and `mtp` GGUFs | 61.6–86.1 GiB | For llama.cpp. Text, image and audio at most: llama.cpp has no native path for MiMo video. Never loaded |
|
| 73 |
-
|
| 74 |
-
Nothing has been measured, so neither repo can be recommended on quality or speed.
|
| 75 |
-
|
| 76 |
-
## Quickstart
|
| 77 |
-
|
| 78 |
-
No inference command has been run against this checkpoint. All that is known to work is downloading it:
|
| 79 |
|
| 80 |
-
``
|
| 81 |
-
hf download patrickbdevaney/MiMo-V2.6-Flash-REAP50 --local-dir MiMo-V2.6-Flash-REAP50
|
| 82 |
-
```
|
| 83 |
|
| 84 |
-
|
| 85 |
|
| 86 |
-
|
|
|
|
|
|
|
|
|
|
| 87 |
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
-
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
## Hardware & memory
|
| 98 |
-
|
| 99 |
-
- **Produced on:** one NVIDIA Jetson AGX Thor (128 GB unified memory). That machine ran the calibration pass, the expert selection and the checkpoint write.
|
| 100 |
-
- **Run on:** nothing. The model has not been loaded for inference anywhere.
|
| 101 |
-
- **Fit estimate (arithmetic only).** The 86.1 GiB of weights plus an FP8 KV cache at the full 1,048,576 tokens comes to about 97.4 GiB before runtime workspace. The KV figure is about 11.25 GiB: 9 full-attention layers × 4 KV heads × (192 + 128) dims × 1 byte, about 11.5 KB per token, while the 39 SWA layers hold only a 128-token window. That total sits under the roughly 117 GiB usable on Thor, but it has not been tried. A BF16 KV cache would double the KV term.
|
| 102 |
-
|
| 103 |
-
## How it was made
|
| 104 |
-
|
| 105 |
-
1. **Calibration.** Expert statistics were accumulated over **49.0M tokens** in 27 chunks of packed 4,096-token rows. Every document was truncated at 4,095 tokens. The pass ran layer by layer through the unpruned model, one layer resident at a time. The corpus has nine domains:
|
| 106 |
-
|
| 107 |
-
| domain | tokens | | domain | tokens |
|
| 108 |
-
|---|---:|---|---|---:|
|
| 109 |
-
| agentic | 11,501,033 | | image | 3,001,238 |
|
| 110 |
-
| ballast (general) | 10,003,073 | | science | 2,980,200 |
|
| 111 |
-
| code | 9,503,748 | | audio | 1,502,892 |
|
| 112 |
-
| math | 7,502,972 | | video | 1,502,610 |
|
| 113 |
-
| finance | 1,503,455 | | | |
|
| 114 |
-
|
| 115 |
-
The image, audio and video domains are real media run through the model's own towers and processor (with their captions), 6.0M tokens in all, rather than text captions alone. Sources were limited to permissively licensed datasets, with licences checked against the Hugging Face API; a source with no declared licence was excluded. Non-English coverage was not measured separately.
|
| 116 |
-
|
| 117 |
-
2. **Selection.** Per-expert saliency used the REAP criterion (`reap_1_1_1`, router-weighted expert activation; [Lasby et al., 2025](https://arxiv.org/abs/2510.13999)). The prune set was solved with the **HOPE** objective (arXiv 2609.18916). HOPE minimises `pᵀFp`, the output error of a prune set, interaction terms between experts included; REAP corresponds to the same objective with those off-diagonal terms set to zero. The ratio was 0.50, uniform across all 47 MoE layers (128 kept per layer). The top 8% of experts in each domain were protected from pruning (`protect_frac 0.08`), and candidates were compared by their **worst** domain rather than their mean.
|
| 118 |
|
| 119 |
-
|
|
|
|
| 120 |
|
| 121 |
-
|
| 122 |
-
|---|---:|---|---|---:|
|
| 123 |
-
| audio (worst) | 0.9935 | | finance | 0.9974 |
|
| 124 |
-
| image | 0.9937 | | ballast | 0.9976 |
|
| 125 |
-
| video | 0.9938 | | code | 0.9983 |
|
| 126 |
-
| science | 0.9973 | | agentic | 0.9983 |
|
| 127 |
-
| math | 0.9974 | | **mean** | **0.9964** |
|
| 128 |
|
| 129 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 130 |
|
| 131 |
-
|
| 132 |
|
| 133 |
-
|
|
|
|
| 134 |
|
| 135 |
-
|
|
|
|
| 136 |
|
| 137 |
## Limitations
|
| 138 |
|
| 139 |
-
-
|
| 140 |
-
|
| 141 |
-
-
|
| 142 |
-
|
| 143 |
-
-
|
| 144 |
-
|
| 145 |
-
## Changelog / corrections
|
| 146 |
-
|
| 147 |
-
- **2026-09-25:** checkpoint published (weights unchanged since then).
|
| 148 |
-
- **2026-10-24:** correction: never served or benchmarked; "~5 tok/s" was bandwidth arithmetic; the 1M-context fit is sizing arithmetic; per-domain retention is a selection-time proxy. No weights were changed.
|
| 149 |
-
- **2026-11-06:** card rewritten and verified against repo files and build artifacts; a 2026-10-26 automated rewrite contained errors.
|
| 150 |
-
|
| 151 |
-
## License & attribution
|
| 152 |
-
|
| 153 |
-
MIT, inherited from the base model. The base model, its architecture, its multimodal towers and its MTP layers are the work of the Xiaomi MiMo team: [`XiaomiMiMo/MiMo-V2.6-Flash-RL`](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL). This is an independent community derivative, not affiliated with or endorsed by Xiaomi. The expert-saliency criterion is REAP ([arXiv 2510.13999](https://arxiv.org/abs/2510.13999)); the prune-set objective is HOPE (arXiv 2609.18916).
|
| 154 |
|
| 155 |
-
##
|
| 156 |
|
| 157 |
-
|
| 158 |
-
|
| 159 |
-
title = {MiMo-V2.6-Flash-REAP50: 50\% routed-expert pruning of MiMo-V2.6-Flash-RL},
|
| 160 |
-
author = {Patrick Devaney},
|
| 161 |
-
year = {2026},
|
| 162 |
-
howpublished = {\url{https://huggingface.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50}}
|
| 163 |
-
}
|
| 164 |
-
```
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
base_model: XiaomiMiMo/MiMo-V2.6-Flash-RL
|
|
|
|
| 4 |
library_name: transformers
|
|
|
|
| 5 |
tags:
|
|
|
|
| 6 |
- moe
|
| 7 |
- pruned
|
|
|
|
| 8 |
- reap
|
| 9 |
- hope
|
| 10 |
+
- mimo_v2
|
|
|
|
| 11 |
- multimodal
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
---
|
| 13 |
|
| 14 |
# MiMo-V2.6-Flash-REAP50
|
| 15 |
|
| 16 |
+
`XiaomiMiMo/MiMo-V2.6-Flash-RL` with **50% of its routed experts removed** — 256
|
| 17 |
+
experts per layer down to **128** — so that it fits and serves on a single
|
| 18 |
+
NVIDIA Jetson AGX Thor (117 GiB unified memory). 86.1 GiB across 65 shards.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
|
| 20 |
+
Vision, audio and video input are preserved; `audio_tokenizer/` ships with the checkpoint.
|
|
|
|
|
|
|
| 21 |
|
| 22 |
+
## How the experts were chosen
|
| 23 |
|
| 24 |
+
Not by activation frequency. Expert saliency was accumulated over a calibration corpus and the
|
| 25 |
+
prune set was solved with **HOPE**, which minimises the output error a prune set actually causes
|
| 26 |
+
including the *interaction* terms between experts — REAP is the same objective with the
|
| 27 |
+
off-diagonal zeroed, and that off-diagonal cannot be recovered after the pass.
|
| 28 |
|
| 29 |
+
| setting | value |
|
| 30 |
+
|---|---|
|
| 31 |
+
| objective | `hope` |
|
| 32 |
+
| saliency criterion | `reap_1_1_1` |
|
| 33 |
+
| prune ratio | 0.50, uniform across layers |
|
| 34 |
+
| per-domain protection | top 8% of every domain held out of the prune set |
|
| 35 |
+
| worst domain retained | 0.9935 (audio) |
|
| 36 |
+
| mean retained | 0.9964 |
|
| 37 |
+
| HOPE objective pᵀFp | 0.01624 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
|
| 39 |
+
Selection is scored per domain and ranked by the **worst** one, never the mean: an average is
|
| 40 |
+
how a criterion that destroys one capability outscores one that preserves all of them.
|
| 41 |
|
| 42 |
+
### Retained gated output mass, by calibration domain
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
|
| 44 |
+
| domain | retention |
|
| 45 |
+
|---|---|
|
| 46 |
+
| audio | 0.9935 |
|
| 47 |
+
| image | 0.9937 |
|
| 48 |
+
| video | 0.9938 |
|
| 49 |
+
| science | 0.9973 |
|
| 50 |
+
| math | 0.9974 |
|
| 51 |
+
| finance | 0.9974 |
|
| 52 |
+
| ballast | 0.9976 |
|
| 53 |
+
| code | 0.9983 |
|
| 54 |
+
| agentic | 0.9983 |
|
| 55 |
|
| 56 |
+
## Routers
|
| 57 |
|
| 58 |
+
Pruning an expert leaves its router column behind. The routers were refitted by output matching
|
| 59 |
+
against the unpruned teacher, routers only, every expert frozen: 47 routers refitted; all 47 kept at the teacher weights (the fit did not beat the baseline).
|
| 60 |
|
| 61 |
+
A refit that failed to beat the untouched baseline was discarded in favour of the baseline, so
|
| 62 |
+
no router here is worse than simply slicing the teacher's.
|
| 63 |
|
| 64 |
## Limitations
|
| 65 |
|
| 66 |
+
- Calibration was English/Chinese text, code, math, science, finance, agentic traces, and
|
| 67 |
+
image/audio/video captions. Domains outside that mix were not measured.
|
| 68 |
+
- The `dflash/` speculative-decoding draft head from the source repo is **not** included: it was
|
| 69 |
+
trained against the unpruned expert set and is not valid for this checkpoint.
|
| 70 |
+
- Pruned MoE experts do not come back. This is a lossy, irreversible transform of the base model.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
|
| 72 |
+
## Provenance
|
| 73 |
|
| 74 |
+
Produced by [patrickbdevaney/xiaomi-2.6-flash-REAP](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP)
|
| 75 |
+
on a single Jetson AGX Thor. MIT, inherited from the base model — attribution to Xiaomi MiMo.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|