AX-Qwen3-Embedding-4B-MLX-AXQ-4bit

An AXQuant (AXQ) mixed-precision MLX checkpoint for Apple Silicon, converted directly from the BF16 source model. The language path is quantized under AXQuant protection floors (embeddings, norms, and other protected tensors remain higher precision).

Development evidence — not a certified AXQuant release. This package has conversion and artifact-integrity records, but it does not publish measured quality, long-context, kernel-speed, or MTP-speed evidence. Do not interpret the AXQ product label as a benchmark claim.

Model details

Property	Value
Base model	Qwen/Qwen3-Embedding-4B
Source revision	unrecorded
Product family	qwen3
Source architecture	Qwen3ForCausalLM (dense); text path optimized
Main-model parameters	4.02B logical parameters
Quantizer	AXQuant 1.0.1
Hub budget class	4bit
AXQuant base precision class	4bit
Planned storage-adjusted BPW	4.8900
Measured main-model BPW	4.8902
Measured total BPW	4.8902
Safetensors weight size	2.46 GB
Approximate complete download	2.47 GB
Configured maximum context	40,960 tokens; practical limits depend on unified memory
Primary runtime	AX Engine, compatibility level A
Compatible runtime	MLX-LM standard text inference, compatibility level B
MTP present	False
Vision sidecar present	False

This repository contains MLX Safetensors. It does not contain PyTorch or GGUF weights.

Choosing an AXQ pack

AXQ names describe a storage-budget product class, not one uniform precision applied to every tensor. Protected tensors remain at higher precision, so the exact measured BPW is authoritative. In particular, a 6bit-named mixed plan may retain 4bit as its base precision while selecting 6-bit, 8-bit, or BF16 for other tensors to meet an approximately 6-BPW total budget. Protection floors can also raise a 4bit-named pack close to (or above) a 6bit budget on small or heavily protected models.

Sibling	Intended trade-off
4bit sibling	Lower-storage AXQ budget; check its exact BPW
6bit sibling	Higher average precision near a 6-BPW budget

See the AutomatosX MLX model catalog for related MLX and OptiQ alternatives.

Download

python -m pip install -U huggingface_hub
hf download AutomatosX/AX-Qwen3-Embedding-4B-MLX-AXQ-4bit --local-dir ./AX-Qwen3-Embedding-4B-MLX-AXQ-4bit

Allow at least 2.47 GB of free disk space. Pin the resulting Hub commit in reproducible deployments rather than relying indefinitely on main.

Run with MLX-LM

python -m pip install -U mlx-lm
mlx_lm.generate \
  --model AutomatosX/AX-Qwen3-Embedding-4B-MLX-AXQ-4bit \
  --prompt "Explain mixed-precision quantization in three sentences." \
  --max-tokens 128 \
  --temp 0.0

MLX-LM compatibility covers standard text/backbone inference. It may ignore AXQuant runtime metadata and optional sidecars (vision.safetensors, mtp.safetensors); this command therefore does not establish MTP acceleration or vision-language quality. The artifact records MLX 0.32.0 and MLX-LM 0.31.3 from conversion.

Serve with AX Engine

After installing AX Engine, download the complete repository and serve the local directory:

ax-engine serve ./AX-Qwen3-Embedding-4B-MLX-AXQ-4bit --port 31418

AX Engine is the authority for the AXQ runtime contract. This development package does not claim runtime speedups until identical-checkpoint benchmarks are published. The artifact records AX Engine version not recorded. Native model-manifest.json status: not included.

Quantization layout

Main-weight precision	Parameters	Share
4bit	3.63B	90.34%
8bit	388.26M	9.65%
bf16	196,096	0.00%

• Quantization methods: affine, bf16.
• Group sizes used by quantized assignments: 32, 64.
• MTP sidecar: not included.
• Vision sidecar: not included.
• Optimization scope: text-path.
• Support tier: convertible.

BF16 sidecars, when present, are included in total download size. Their presence does not by itself establish MTP acceleration or vision-language quality.

Evidence and validation status

Check	Status
Planning evidence	architecture_prior
Calibration	none; the allocation is based on architecture priors
Quantizer execution	253/253 recorded module conversions succeeded; 0 fallbacks
AX Engine native manifest	not included
Quality versus BF16 or uniform baselines	Not published; no quality-retention claim
MTP acceptance and speed	not measured; no MTP speedup claim
AX Engine kernel evidence	unmeasured
Vision-language quality	Not applicable (no vision sidecar in this package)
Long-context quality	40,960-token capacity is config metadata, not a validated claim
Release certification	Not certified; formal AXQuant M0-M8 gates are not closed

Intended use and limitations

• Intended for local development and evaluation on Apple Silicon with MLX-compatible runtimes.
• No minimum unified-memory figure is claimed; loadability depends on model size, context length, KV-cache policy, runtime buffers, and other processes using unified memory.
• Architecture-prior allocation is not measured sensitivity. It must not be presented as measured model quality.
• The configured context window can require substantially more memory as the KV cache grows.
• Upstream capabilities, limitations, biases, and responsible-use guidance still apply.

Provenance and audit files

• axquant_manifest.json: package identity, byte accounting, runtime contract, software versions, and file checksums.
• axquant_plan.json: per-tensor precision decisions and planning evidence.
• axquant_quantizer_execution.json: conversion coverage and fallback records.
• axquant_runtime.json: AX Engine and MLX-LM compatibility contract.

All published provenance uses repository-relative paths. Local source paths are stripped before publication. The checkpoint was converted from BF16 rather than re-quantized from an OptiQ artifact. Parallel OptiQ repositories use a different quantizer and should not be assumed to have identical BPW or quality.

License

The checkpoint follows the upstream model license where applicable (often Apache License 2.0). See the Qwen/Qwen3-Embedding-4B model card for license terms, model limitations, and responsible-use guidance.