Vontra/Ling-3.0-flash-oQ4

Ling 3.0 Flash — oQ4

A native Apple-silicon conversion of inclusionAI/Ling-3.0-flash, quantized with oMLX's optimized mixed-precision oQ4 recipe and packaged for oMLX and compatible MLX-LM runtimes.

Original model · InclusionAI on ModelScope · OpenRouter · MLX-LM

About this conversion

This repository contains an oMLX-optimized mixed-precision oQ4 conversion of Ling 3.0 Flash. Ling is a 124B-total / 5.1B-active hybrid-linear mixture-of-experts model. The conversion preserves the upstream tokenizer and chat template and includes the custom MLX architecture adapter required for Kimi Delta Attention, gated MLA, and sparse MoE layers.

Item	Value
Base model	inclusionAI/Ling-3.0-flash
Format	MLX safetensors
Quantization	oQ4 affine mixed precision, group size 64
Base precision	4-bit
Protected modules	116 at 6-bit and 339 at 8-bit
Conversion/runtime stack	oMLX with MLX-LM 0.31.3 / MLX 0.32.0
Weight shards	15
Weight size	71.55 GB (66.63 GiB)
Maximum configured context	262,144 tokens
Architecture	bailing_hybrid

This model uses the included bailing_hybrid.py custom MLX adapter. In oMLX, enable Trust Remote Code for this model before loading it. Standalone MLX-LM requires a build whose mlx_lm.load supports the trust_remote_code argument and repository-provided model_file adapters; stock PyPI mlx-lm 0.31.3 does not provide that loader path.

Apple-silicon performance

This checkpoint was load-tested, generation-tested, and benchmarked on:

Hardware	Configuration
Host	Mac Studio
Chip	Apple M3 Ultra
CPU	32 cores (24 performance + 8 efficiency)
Unified memory	256 GB
Runtime	oMLX-bundled MLX-LM 0.31.3 / MLX 0.32.0

A warmed local test produced:

Measurement	Result
Decode (median)	74.04 tokens/s
Individual decode runs	74.04 / 69.31 / 74.15 tokens/s
Reported peak memory	72.37 GB
Timed runs	3 × 256 generated tokens
Warm-up	256 generated tokens
Prompt	39 tokens after chat templating

The decode figure is the median of three greedy 256-token runs after a full 256-token Metal-kernel warm-up. It is a practical local reference, not a controlled cross-platform benchmark. Prompt length, context growth, sampler settings, memory pressure, thermal state, and runtime versions can materially change performance.

Recommended use with oMLX

1. Download the model into the oMLX model directory:

hf download Vontra/Ling-3.0-flash-oQ4 \
  --local-dir ~/.omlx/models/Vontra/Ling-3.0-flash-oQ4

2. Refresh the oMLX model registry.
3. Open the model settings and enable Trust Remote Code.
4. Load Ling-3.0-flash-oQ4 and use the normal chat UI or OpenAI-compatible endpoint.

Example request:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OMLX_API_KEY" \
  -d '{
    "model": "Ling-3.0-flash-oQ4",
    "messages": [{"role": "user", "content": "Explain why hybrid linear attention is useful."}],
    "temperature": 0.6,
    "top_p": 0.95,
    "max_tokens": 512
  }'

Thinking mode is enabled by the upstream chat template by default. Disable it with the model's chat-template configuration when a direct answer is preferred.

Compatible standalone MLX-LM builds

With an MLX-LM build that supports repository-provided model adapters:

mlx_lm.generate \
  --model Vontra/Ling-3.0-flash-oQ4 \
  --trust-remote-code \
  --prompt "Explain why hybrid linear attention is useful." \
  --max-tokens 512 \
  --temp 0.6 \
  --top-p 0.95 \
  --top-k 20

Python usage with the same capability:

from mlx_lm import load, generate

model, tokenizer = load(
    "Vontra/Ling-3.0-flash-oQ4",
    trust_remote_code=True,
    tokenizer_config={"trust_remote_code": True},
)
messages = [
    {"role": "user", "content": "Explain sparse mixture-of-experts routing."}
]
prompt = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=False,
    enable_thinking=False,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))

Only enable remote code after reviewing the included adapter files.

Architecture

Ling 3.0 Flash alternates Kimi Delta Attention (KDA) and gated Multi-head Latent Attention (MLA) in a 5:1 ratio and uses highly sparse routed experts.

Architecture detail	Upstream value
Total / active parameters	124B / 5.1B
Transformer layers	35 KDA + 7 gated MLA
Dense layers	2
Routed / shared experts	512 / 1
Active routed experts	8
Attention heads	32
Hidden size	2,560
Expert intermediate size	768
Dense intermediate size	6,144
Vocabulary size	157,184
Context training schedule	8K → 32K → 256K

The included MLX adapter uses MLX-LM primitives for delta attention, absorbed MLA projections, RoPE, and quantized SwitchGLU experts. The auxiliary MTP training head is excluded from ordinary causal generation; this release does not claim MTP or DSpark speculative-decoding support.

Upstream model highlights

InclusionAI describes Ling 3.0 Flash as a hybrid reasoning model for software-engineering agents, tool use, deep research, general knowledge, mathematical reasoning, instruction following, and long-context understanding.

The upstream defaults are:

thinking: enabled
temperature: 0.6
top_p: 0.95
top_k: 20

For benchmark methodology, scores, intended use, limitations, and framework-specific deployment instructions, see the original InclusionAI model card.

Conversion and validation notes

• Source weights: upstream BF16 checkpoint.
• Quantization mode: affine, group size 64.
• oQ4 uses a 4-bit base with 455 sensitive modules kept at higher precision.
• Both quantization and quantization_config preserve the per-module recipe.
• The upstream chat template is included unchanged.
• All 2,067 converted tensors and all 15 indexed shards were checked locally.
• The custom adapter was loaded with explicit trust and exercised through end-to-end generation.
• Quantization can reduce output quality relative to BF16; use a higher-precision variant when quality matters more than memory use.

This is a community conversion, not an official InclusionAI release. Validate quality and numerical behavior on representative workloads before production use.

License and attribution

The upstream model declares the MIT License in its Hugging Face metadata. A standard MIT licence copy is included in this repository.

All model design, training, benchmark, and upstream documentation credit belongs to InclusionAI and the original contributors. The mixed-precision conversion, Apple-silicon validation, compatibility packaging, and model card are provided by Vontra.