OsaurusAI/Raptor-8B-A1B-preview-MXFP8

OsaurusAI/Raptor-preview

Preview build. MXFP8 quantization of Raptor-8B-A1B — a Liquid lfm2_moe hybrid — for Apple Silicon / MLX. Loads with stock mlx_lm, no custom code.

⚠️ PREVIEW / NOT A VALIDATED RELEASE. The source package is marked INTERNAL_STABLE_PUBLIC_RELEASE_NOT_AUTHORIZED and carries its own warnings: AIME / HLE / Terminal-Bench / SWE-bench scores are incomplete, and no sub-10 GB quantization had previously passed behaviour parity. This bundle is 8.2 GB. Measured quality is stated plainly below — read it before relying on this for anything.

Architecture

Field	Value
Base	Raptor-8B-A1B (Lfm2MoeForCausalLM, model_type: lfm2_moe)
Layout	24 layers, hybrid — LIV short-conv blocks + 6 full-attention
MoE	32 experts, 4 active per token (moe_intermediate 1792), 2 dense layers
Hidden	2048, vocab 128000, tied embeddings
Context	128K
On disk	8.2 GiB (8.252 bits/weight)

Quantization

Uniform MXFP8 (e4m3 codes + e8m0 scales, group size 32) across all 133 quantizable modules — including all 66 stacked expert tensors and the 36 short-conv input/output projections.

fp16 passthrough, deliberately:

• the 18 depthwise short-conv kernels (conv.conv) — not matmuls
• the 22 MoE routers (feed_forward.gate) — 32-way selection
• all norms

MXFP8 is structural: no Hessian allocation, no AWQ, no imatrix refit. It is the uncalibrated tier by design.

Measured quality

Against the bf16 source, 12 held-out prompts, scored on the generated span:

Metric	Value
Median KL	0.0629 nats
Mean KL	0.0679 nats
Top-1 agreement	90.28 %

Be aware what that means: roughly 1 token in 10 differs from bf16. This is weaker than MXFP8 on larger dense models (Qwen3.8-27B MXFP8 scores 0.0189 / 95.83 %), which is consistent with an A1B — only ~1 B active parameters means far less redundancy to absorb quantization error. A calibrated affine tier (AWQ + imatrix) would likely beat this at smaller size and is not yet built.

Verified working

• Reasoning: emits <think>…</think> correctly, on by default
• Tool calling validated by the model's own shipped contract parser (lfm25_contract.py), which is included in this bundle:

<|tool_call_start|>[get_weather(city="Osaka")]<|tool_call_end|>
  -> ToolCall(name='get_weather', arguments={'city': 'Osaka'})

Serving contract (stamped in jang_config.json)

Taken from the vendor's own raptor_generation_profiles.json, which ships in this bundle — not invented.

• 🚨 The vendor default is GREEDY (do_sample: false), not a temperature. Sampling keys are written absent rather than as inactive values.
• Stamped default is the vendor's interactive_reasoning profile: greedy, reasoning ON, max_new_tokens 8192.
• All seven vendor profiles are carried as sampling_modes, including swe_bench_lfm_adapted (T=1.0, top_p=0.95) and terminal_bench_2_1_ornith_target (T=1.0, top_p=1.0) for agentic use. liquid_native_diagnostic_only is flagged diagnostic — never a serving default.
• Tools: lfm25_native_python_literal, <|tool_call_start|> / <|tool_call_end|>. Native calls look Python-like but are data — parsed by a small AST subset, never evaluated. Do not .strip() string argument values; a trailing newline is content.
• Stop on <|im_end|> (124900).
• Runtime family lfm2 (cache_type: hybrid, cache_subtype: lfm2_moe_hybrid_ssm).

Use it

pip install -U mlx-lm

from mlx_lm import load, generate
model, tok = load("OsaurusAI/Raptor-preview")
p = tok.apply_chat_template([{"role":"user","content":"Name three primes under 20."}],
                            add_generation_prompt=True, tokenize=False)
print(generate(model, tok, p, max_tokens=256, verbose=False))

Credits

Quantized by Jinho Jang — eric@osaurus.ai