File size: 2,600 Bytes
b23eaff | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 | ---
library_name: mlx
license: apache-2.0
license_link: https://huggingface.co/internlm/Intern-S2-Mobius/blob/main/LICENSE
pipeline_tag: text-generation
base_model: internlm/Intern-S2-Mobius
tags:
- mlx
- mlx-lm
- quantization
- 4-bit
---
# Intern-S2-Mobius (4-bit MLX)
An MLX **4-bit** (affine, group size 64) quantization of
[internlm/Intern-S2-Mobius](https://huggingface.co/internlm/Intern-S2-Mobius) — a 35B hybrid model:
Gated-DeltaNet linear attention / full attention at interval 4, with 2560 experts in 4 globally-shared
routed MoE banks. Runs on Apple Silicon.
- Base model: [internlm/Intern-S2-Mobius](https://huggingface.co/internlm/Intern-S2-Mobius) (Apache-2.0).
- Quantization: 4-bit affine, group size 64; shared-expert / router gates kept at 8-bit. ~19.6 GB peak, ~4.5 bpw.
- Architecture: `interns2_mobius` (`text_config.model_type = interns2_mobius_text`), 40 layers, `head_dim` 256,
MoE 2560 experts / top-8, `num_blocks` 4. `max_position_embeddings` 262144.
## Quick start
Requires the `mlx-lm` build that ships the `interns2_mobius` architecture
(official part of `mlx-lm` as of the model-support PR):
```bash
pip install -U mlx-lm
mlx_lm.generate --model nightscape/Intern-S2-Mobius-4bit-mlx \
-p "The secret to baking a good cake is" -m 1024 --trust-remote-code
```
`--trust-remote-code` is mandatory: the checkpoint bundles a custom tokenizer
(`tokenization_interns1.py`) and model code.
## Details
- This is an MLX conversion; `transformers` does not yet ship `interns2_mobius`. The reference is the
upstream repo's `trust_remote_code` implementation, verified by full bf16 logit diff (argmax agreement
38/39, the sole miss a bit-identical tie).
- Capability spot-check (this 4-bit conversion): MMLU-Pro 88.3% ± 4.1 (n=60) vs upstream bf16 89.05;
GSM8K 97–98% (n=100). Short-generation-budget evals truncate chain-of-thought and depress scores — an
eval artifact, not a capability drop.
- Text-only: this conversion is the language model. The upstream checkpoint is tagged `image-text-to-text`,
but the MLX port loads the language model and generates text (no vision tower on this path).
## Companion
An experimental MTP (Multi-Token-Prediction) head is published separately:
[nightscape/Intern-S2-Mobius-4bit-mlx-mtp](https://huggingface.co/nightscape/Intern-S2-Mobius-4bit-mlx-mtp).
It is consumed by the `omlx` server's `interns2_mobius` MTP driver, **not** by stock `mlx-lm`.
## License
Weights and code under Apache-2.0 — see [LICENSE](LICENSE). Model by InternLM; this is a derivative
conversion of their weights plus the MLX port.
|