---
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3-Coder-Next/blob/main/LICENSE
pipeline_tag: text-generation
base_model: Qwen/Qwen3-Coder-Next
base_model_relation: quantized
quantization: exl3
library_name: exllamav3
---

Quantization was performed using [exllama3 v0.0.20](https://github.com/turboderp-org/exllamav3).

> **Note:** In exllamav3 v0.0.21, there were [fixes to the Qwen3-Next inference pipeline](https://github.com/turboderp-org/exllamav3/commit/d3e02500e0dac2d67ca7fc9babed5d40dcf33689). These quants still work fine, but with v0.0.21+ they should perform even better. It is recommended to use exllamav3 v0.0.21 or later for best results.

| Quant | Size (GB) | KL-div (quant, orig) | KL-div (orig, quant) | Perplexity | Top-K K=1 | Top-K K=2 | Top-K K=3 | Top-K K=4 | Top-K K=5 |
|---|---|---|---|---|---|---|---|---|---|
| [2.0bpw](https://huggingface.co/NeuroSenko/Qwen3-Coder-Next-exl3/tree/2.0bpw) | 20 | 0.52142615 | 0.52278535 | 23.73415073 | 0.6961 | 0.3484 | 0.1402 | 0.0498 | 0.0167 |
| [3.0bpw](https://huggingface.co/NeuroSenko/Qwen3-Coder-Next-exl3/tree/3.0bpw) | 29 | 0.24568403 | 0.24622221 | 20.58547252 | 0.7866 | 0.4894 | 0.2579 | 0.1190 | 0.0513 |
| [4.0bpw](https://huggingface.co/NeuroSenko/Qwen3-Coder-Next-exl3/tree/4.0bpw) | 38 | 0.15672405 | 0.15667850 | 19.63543922 | 0.8338 | 0.5783 | 0.3511 | 0.1923 | 0.0990 |
| [5.0bpw](https://huggingface.co/NeuroSenko/Qwen3-Coder-Next-exl3/tree/5.0bpw) | 47 | 0.12297954 | 0.12280908 | 19.81022066 | 0.8562 | 0.6287 | 0.4088 | 0.2463 | 0.1388 |
| [6.0bpw](https://huggingface.co/NeuroSenko/Qwen3-Coder-Next-exl3/tree/6.0bpw) | 57 | 0.10448053 | 0.10464503 | 19.88056610 | 0.8707 | 0.6590 | 0.4502 | 0.2848 | 0.1704 |
| [7.0bpw](https://huggingface.co/NeuroSenko/Qwen3-Coder-Next-exl3/tree/7.0bpw) | 66 | 0.10106506 | 0.10081614 | 19.61846442 | 0.8730 | 0.6666 | 0.4614 | 0.2983 | 0.1821 |
| [8.0bpw](https://huggingface.co/NeuroSenko/Qwen3-Coder-Next-exl3/tree/8.0bpw) | 75 | 0.13291914 | 0.13419860 | 19.85572412 | 0.8631 | 0.6503 | 0.4468 | 0.2885 | 0.1771 |
| original | 148 | - | - | 19.78538866 | - | - | - | - | - |

## Tool Calls Support for Qwen/GLM Models

The official tabbyAPI doesn't support tool calls for Qwen and GLM models yet.

If you're using Qwen-Code, OpenClaw, or similar software that need tool call support, you can use [my fork](https://github.com/NeuroSenko/tabbyAPI/tree/tools-support) with the `tools-support` branch:

**Clone directly:**
```bash
git clone -b tools-support https://github.com/NeuroSenko/tabbyAPI
```

**Or add to existing tabbyAPI installation:**
```bash
git remote add neurosenko https://github.com/NeuroSenko/tabbyAPI
git fetch neurosenko
git checkout -b tools-support neurosenko/tools-support
```

This branch includes native tool calling support for Qwen and GLM model families.