File size: 2,023 Bytes
3b16649 328a497 65bb2b2 3b16649 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 | ---
license: mit
base_model: inclusionAI/Ling-3.0-tiny
library_name: mlx
tags:
- mlx
- ling
- bailing_hybrid
- moe
- apple-silicon
pipeline_tag: text-generation
---
# Ling-3.0-tiny — MLX 4-bit
The first MLX conversion of [inclusionAI/Ling-3.0-tiny](https://huggingface.co/inclusionAI/Ling-3.0-tiny):
a 7.9B-total / **1.3B-active** sparse-MoE reasoner (128 experts, top-8 + 1 shared)
with a KDA + MLA hybrid attention stack and 131K context, MIT licensed.
**4.2 GB at 4.507 bits/weight** — it fits and runs on an 8 GB Apple Silicon Mac.
| | |
|---|---|
| Quantization | 4-bit, group size 64 (router kept 8-bit, short-conv weights fp) |
| Size on disk | 4.2 GB |
| Context | 131,072 tokens |
| Active parameters | 1.3B per token |
| License | MIT (inherited from the base model) |
## Serve it
The `bailing_hybrid` architecture is not in upstream `mlx-lm` yet — this
checkpoint is served by [rapid-mlx](https://rapidmlx.com), which ships a
verified native implementation (reference parity 1.5e-6 against the
official modeling code):
```bash
pip install -U rapid-mlx # 0.12.10 or newer
rapid-mlx serve ling-3.0-tiny-4bit
```
You get an OpenAI-compatible server on `localhost:8000` with reasoning
(`reasoning_content`) and tool calling parsed natively — thinking is
controlled with `chat_template_kwargs: {"enable_thinking": true}` or the
model's `detailed thinking on/off` system-prompt switch.
Once `mlx-lm` gains native `bailing_hybrid` support, this checkpoint will
load there unchanged.
## Conversion provenance
Converted with `mlx_lm.convert` (quantize=True, q_bits=4, q_group_size=64)
running rapid-mlx's vendored `bailing_hybrid` implementation
([PR #1817](https://github.com/raullenchai/Rapid-MLX/pull/1817)), which was
verified against the official `modeling_bailing_moe_v3.py` on identical
random weights to a max logits deviation of 1.5e-6 (full prefill) /
1.9e-6 (token-by-token incremental) before conversion. End-to-end
chat / reasoning / tool-call behaviour validated on an M2 Pro Mac mini.
|