cudo528/Nanbeige4.2-3B-mlx-bf16

Nanbeige4.2-3B — MLX (bfloat16)

An MLX bfloat16 conversion of Nanbeige/Nanbeige4.2-3B, ready to run on Apple Silicon with mlx-lm.

• Format: MLX safetensors, bfloat16 (no quantization — full-precision weights)
• Size: ~8.3 GB · ~4.2B stored parameters
• Converted from: Nanbeige/Nanbeige4.2-3B with mlx_lm.convert(..., quantize=False, dtype="bfloat16")

This model needs a custom architecture file to load. Nanbeige uses a looped transformer architecture (model_type: "nanbeige") that mlx-lm does not support out of the box. Without the step below you will get: ValueError: Model type nanbeige not supported. This repo ships the required nanbeige.py — see Setup.

---

Setup

Install mlx-lm, then drop the bundled nanbeige.py into mlx-lm's model registry:

pip install -U mlx-lm

# Download nanbeige.py from this repo, then copy it into mlx_lm/models/:
python - <<'PY'
import mlx_lm, os, shutil
dst = os.path.join(os.path.dirname(mlx_lm.__file__), "models", "nanbeige.py")
shutil.copy("nanbeige.py", dst)
print("installed ->", dst)
PY

Note: re-installing or upgrading mlx-lm removes this file — just copy it again.

---

Usage

Python

from mlx_lm import load, generate

model, tokenizer = load("cudo528/Nanbeige4.2-3B-mlx-bf16")

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "What is the capital of France?"}],
    add_generation_prompt=True,
)

text = generate(model, tokenizer, prompt=prompt, max_tokens=256, verbose=True)

Command line

mlx_lm.generate \
  --model cudo528/Nanbeige4.2-3B-mlx-bf16 \
  --prompt "Explain rotary position embeddings in one paragraph."

---

Model details

Base model	Nanbeige/Nanbeige4.2-3B
Architecture	nanbeige — Llama-style decoder run as a looped transformer
Decoder layers	22 (weights) · executed 2× (num_loops = 2) → 44 effective passes
Hidden size	3072
Attention heads	48 (head_dim = 128)
KV heads	8 (grouped-query attention)
Intermediate size	10752 (SwiGLU / SiLU)
Normalization	RMSNorm (eps = 1e-5)
Position encoding	RoPE (theta = 7e7)
Vocab size	166,144
Max context	262,144
Tied embeddings	No (separate lm_head)
Weight dtype	bfloat16

About the looped architecture

Nanbeige reuses a single 22-layer stack for num_loops passes instead of stacking distinct layers. The effective forward is:

h = embed(x)
for loop in range(num_loops):            # num_loops = 2
    for layer in layers:  h = layer(h)   # KV cache slot = layer_idx + loop * 22
    h = norm(h)                          # final norm after each loop pass
logits = lm_head(h)

Because layers are shared across loops, the KV cache holds num_loops × num_layers = 44 entries (each loop pass keeps its own keys/values). This behavior is implemented in the bundled nanbeige.py.

---

Conversion

from mlx_lm import convert

convert(
    "Nanbeige/Nanbeige4.2-3B",
    "Nanbeige4.2-3B-mlx-bf16",
    quantize=False,
    dtype="bfloat16",
)

The converted model loads in mlx-lm and produces coherent generations (e.g. correctly answers "The capital of France is Paris."). Weights are cast to bfloat16 with no quantization, so quality matches the original within bf16 precision.

---

Credits & license

• Original model & weights: Nanbeige/Nanbeige4.2-3B by the Nanbeige team.
• This is an unofficial MLX conversion; all credit for the model goes to the original authors.
• Released under Apache-2.0, following the base model's license. Please also review the base model's usage terms.