Instructions to use AutomatosX/AX-DeepSeek-V4-Flash-0731-MLX-AXQ-2bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use AutomatosX/AX-DeepSeek-V4-Flash-0731-MLX-AXQ-2bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("AutomatosX/AX-DeepSeek-V4-Flash-0731-MLX-AXQ-2bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use AutomatosX/AX-DeepSeek-V4-Flash-0731-MLX-AXQ-2bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "AutomatosX/AX-DeepSeek-V4-Flash-0731-MLX-AXQ-2bit"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "AutomatosX/AX-DeepSeek-V4-Flash-0731-MLX-AXQ-2bit" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AutomatosX/AX-DeepSeek-V4-Flash-0731-MLX-AXQ-2bit", "messages": [ {"role": "user", "content": "Hello"} ] }' - Atomic Chat
AX-DeepSeek-V4-Flash-0731-MLX-AXQ-2bit
Development / experimental AXQuant 2-bit pack of
deepseek-ai/DeepSeek-V4-Flash-0731
@ 7872f01b1d1fe23eabc4c98b48bffcef5a386062.
Converted on df-macstudio-m2 (Apple M2 Ultra, 192 GB) from the native FP8
0731 source (quant_method=fp8). Product class 2bit-experimental.
This is not the older
DeepSeek-V4-FlashHub pack. Do not treat AX-DeepSeek-V4-Flash-MLX-AXQ-2bit certificates as evidence for this 0731 revision.
Measured precision
| Property | Value |
|---|---|
| Target class | 2bit-experimental |
| Measured main BPW | 3.1328993873020314 |
| Measured total BPW | 3.2142055528774454 |
| Weight bytes | 122,212,298,775 |
| Source | deepseek-ai/DeepSeek-V4-Flash-0731@7872f01b1d1fe23eabc4c98b48bffcef5a386062 |
| Convert host | df-macstudio-m2 |
| AXQuant | 1.8.1 |
| mlx / mlx-lm | 0.32.0 / 0.31.3 (vendored deepseek_v4 + FP8 load hook) |
Claims
| Claim | Status |
|---|---|
| Converted on Studio from the pinned 0731 revision | Yes |
| mlx-lm load + generate smoke | Passed on df-macstudio-m2 |
Official DSV4 chat_template.jinja |
In pack |
| Checkpoint Tier 1 (generation viability suite) | Not certified on this record; Studio 15+15 factory QA with chat is 0.633 combined |
| AX Engine native manifest | Not generated — generate-manifest --validate rejected split switch_mlp.gate_proj / up_proj ([256, 2048, 256] vs fused [256, 4096, 256]) |
| MTP acceleration | Not certified |
Requires AX_ENGINE_2BIT_EXPERIMENTAL=1 if served with AX Engine after a future
manifest fix. mlx-lm generate does not need that env.
Attribution
Base weights © DeepSeek. Quantization by AXQuant (development).
- Downloads last month
- 132
2-bit
Model tree for AutomatosX/AX-DeepSeek-V4-Flash-0731-MLX-AXQ-2bit
Base model
deepseek-ai/DeepSeek-V4-Flash-0731