limite-mlx

MLX implementation of the limite architecture, packaged so that stock mlx-lm can load Limite checkpoints with no patching.

pip install mlx-lm "limite-mlx @ https://huggingface.co/pierjoe/limite-mlx/resolve/main/limite_mlx-0.1.0-py3-none-any.whl"
mlx_lm.generate --model pierjoe/limite-1b-violetto-mlx-4bit \
  --prompt "What is the remainder when 7^2026 is divided by 13?" \
  --max-tokens 3000 --temp 0.6 --top-p 0.95

Models that use it are in the collection.

Why this exists

mlx-lm resolves an architecture with a single line:

importlib.import_module(f"mlx_lm.models.{model_type}")

and offers no plugin hook, so the usual advice for an unsupported architecture is to copy a file into site-packages/mlx_lm/models/ — which every mlx-lm upgrade deletes. This package appends a one-name finder to sys.meta_path at interpreter startup instead, so that import call resolves without mlx-lm's package directory being touched.

The finder is appended, not prepended, so the day mlx-lm ships its own models/limite.py the official implementation wins and this package goes inert.

What the architecture does differently

Ten deviations from a Llama-style decoder, all pinned in the checkpoint's config.json and reproduced exactly:

• gain-free RMSNorm with torch's dtype-dependent default epsilon (finfo(bfloat16).eps = 0.0078125, not 1e-6)
• attention softmax scale is a literal 0.1, not head_dim ** -0.5
• layers 3, 7, … 47 are full-attention and rotary-free; the rest use a sliding window whose span includes the query token, so 1025 keys rather than 1024
• partial rotary on 64 lanes over adjacent pairs, sine negated on odd lanes, with the cos/sin tables rounded to bfloat16 as part of the trained function
• per-layer qkv_scale / o_scale folded into the projections in bfloat16
• a second embedding table gated into the attention values on 16 layers, applied before QK-norm
• a per-head output gate, 2 * sigmoid(...), before o_proj
• XSA: a float32 cross-head correction removing the component of the attention output along the normalised value vector
• MUDD: at layers 24 and 47 both the attention input and the residual base are input-dependent mixes of earlier residual states, accumulated left to right
• logits squashed through 23 * sigmoid((raw + 5) / 7.5)

Validation

Ported from the reference vLLM plugin and checked against an independent NumPy oracle with bit-level bfloat16 emulation:

• logits vs oracle — argmax identical at every position; max deviation smaller than the architecture's own bfloat16 rounding noise
• 1025-key sliding window — positions straddling the boundary match exactly
• prefill vs incremental decode — argmax agreement at 64 and 1100 tokens
• bf16 conversion vs original checkpoint — bit-identical logits

This architecture is unusually rounding-sensitive: the residual stream reaches |h| ~ 1.5e5 by layer 47, and a single bfloat16 ULP perturbation at layer 1 moves the final logits by 1.26.

License

Apache-2.0, matching the upstream model and reference implementation. The model itself is Paradigma's work; this is the MLX port.