Sage-S2.1-Flash-oQ4e-mtp

A mixed-precision MLX checkpoint of internlm/Sage-S2.1-Flash (Shanghai AI Laboratory / InternLM), a fine-tune of GLM-5.3-Flash with the same architecture: a natively multimodal mixture-of-experts model with 320B total and 18B active parameters.

It was quantized with oQ (oMLX v0.7.4.dev1) from the official FP8 release, internlm/Sage-S2.1-Flash-FP8 at revision 716e6436905650b7be93e1b7dedfb0bb95afd3a3, using the same recipe as Jundot/GLM-5.3-Flash-oQ4e-mtp: oQ4e, with the MTP head preserved.

The effective average is 4.641 bits/weight for the complete checkpoint, the same layout as that GLM pack.

Quantization and Bit Distribution

oQ4e allocates bits by measured layer sensitivity, then rounds each group with an importance matrix (128 samples x 512 tokens) and a weighted least-squares refit of its scale and bias.

One difference from the GLM pack's build: oMLX 0.7.4.dev1 streams this architecture's calibration one decoder layer at a time from the FP8 source, where the GLM pack (oMLX v0.7.3.dev1) calibrated on a temporary 4-bit proxy.

Storage format Logical weights Share of total Tensor storage Effective bits/weight
Affine 4-bit, group 64 311.654B 96.991% 163.266 GiB 4.50
Affine 8-bit, group 64 8.976B 2.793% 8.882 GiB 8.50
BF16 0.607B 0.189% 1.131 GiB 16.00
FP32 0.086B 0.027% 0.322 GiB 32.00
Total 321.323B 100% 173.600 GiB 4.641

Storage Breakdown

Component Logical weights Storage
Language backbone 313.327B 180.966 GB / 168.538 GiB
Vision encoder 0.564B 1.127 GB / 1.050 GiB
MTP head 7.433B 4.309 GB / 4.013 GiB
Total 321.323B 186.402 GB / 173.600 GiB

Notes

  • Serving: built with oMLX 0.7.4.dev1 and served with it on a 256 GB M3 Ultra (MTP on, chunked prefill).
  • Quality: we have not measured this pack against the original FP8 or BF16 weights.
  • Chat template: Sage's own. Unlike GLM-5.3-Flash's, it keeps earlier turns' reasoning in the conversation history, so multi-turn agent prompts are longer.
  • Not included: Sage's scientific-input projectors (molecules, crystals, DNA/RNA, spectra, EEG). They run outside the language model; see the source repository.

License

MIT, following the source model. Sage-S2.1-Flash's code and weights are released under the MIT License by Shanghai AI Laboratory / InternLM. Sage-S2.1-Flash is built on GLM-5.3-Flash, whose MIT license notice (Z.AI) is included as LICENSE.

Downloads last month
71
Safetensors
Model size
321B params
Tensor type
U32
路
F32
路
BF16
路
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 馃檵 Ask for provider support

Model tree for rdbell/Sage-S2.1-Flash-oQ4e-mtp

Quantized
(13)
this model