MiniMaxAI

/

MiniMax-M3.1-Flash

Like 542

Follow

MiniMax 11.7k

We ported MiniMax's M3.1-Flash — a 428-billion-parameter mixture-of-experts
model with ~23 billion active parameters — into our own Blackwell inference
engine. It is the second member of the MiniMax family we run: M3.1 shares M3's
decoder geometry, so the work was to absorb what the new checkpoint changes,
not to write a new architecture.

Two changes matter for an inference engine. First, MiniMax Sparse Attention now
runs on every layer (M3 kept the first three layers dense), so the
Lightning-Indexer block selection is always live; our kernel already handled it
and the switch is entirely data-driven. Second, the release is
quantization-aware-trained and shipped pre-quantized: the dense
projections are MXFP8 (FP8 payload with one shared exponent per 32 columns) and
the routed experts are already NVIDIA NVFP4. The converter dequantizes the
dense side once and re-quantizes it to our uniform NVFP4 format, and copies the
expert records through verbatim — the expert bytes on disk are the bytes the
kernel reads, with no requantization pass.

The 236 GB release becomes a 227 GB tree. On a single 32 GB RTX 5090 the dense
graph is pinned in 10.5 GiB and the routed-expert set streams from disk through
an LRU of device slots. The model answers the canonical fact check correctly
and deterministically ("The capital of France is Paris."), passes our three
structured-answer parity suites, and calls tools with the checkpoint's native
XML format.

What the checkpoint forced into the engine

1. Sparse attention on all 60 layers. The indexer weights exist on layers
   0-2 (they did not in M3), and its key history must be cached on those layers
   too. The loader is now dispatch-driven and each model member cross-checks
   the dispatch arrays against its own frozen passport, so an M3 tree and an
   M3.1 tree cannot be loaded as each other.
2. RoPE theta 1e7 (M3: 5e6), including the indexer's RoPE. The same
   passport check pins it.
3. A pre-quantized source. Dense projections arrive as FP8 with a
   per-32-column power-of-two exponent; routed experts arrive as NVFP4 with a
   per-16 FP8 block scale and a per-tensor divisor. The converter feeds the
   dense pair to the engine's existing MXFP8-aware quantizer and inverts the
   expert divisor to our multiply convention — the same rule we already used for
   another vendor's NVFP4 release.
4. Fixed adaptive thinking. The new chat template removes the thinking-mode
   switch: the model decides for itself whether to reason, and the generation
   prompt no longer carries the "answer directly" marker. In practice the model
   opens a short <mm:think> block and then answers; a parity probe that only
   looks at the first handful of tokens would miss the answer, so the check
   window is adapted.
5. A new system prompt and role mapping. The default system message names
   the model version and appends an opaque <model_id> line, mid-conversation
   system turns render as developer, and consecutive tool results share one
   message block. All three are reproduced by a per-checkpoint render policy, so
   the M3 template remains untouched.
6. Same vocabulary. The tokenizer files are byte-identical to M3's, so the
   already-verified pre-tokenizer policy transfers unchanged.