Accelerating Qwen3.6 on Intel® Core™ Ultra Series 3 with DFlash
Qwen3.6-35B-A3B is a highly efficient Mixture-of-Experts (MoE) model that brings strong coding, reasoning, and agentic capabilities to the AI PC, making it a stepping stone toward local AI.
MoE models are more challenging to accelerate with Speculative Decoding (SD): verifying several drafted tokens can pull in many more experts, dramatically inflating the effective active parameter count.
Running Qwen3.6-35B-A3B with DFlash SD and OpenVINO™ on an Intel® Panther Lake laptop, we achieve an average speedup of 2.2x on HumanEval (90 t/s), 1.3x on MT-Bench and 1.6x on GSM8K.
DFlash also accelerates the dense Qwen3.6-27B and Qwen3.5-9B, with larger gains (up to 5.5× and 3.4× on HumanEval), as dense models aren't subject to the MoE expert-loading penalty.
Code will be available in upcoming OpenVINO.GenAI releases.
Qwen3.6-35B-A3B & MoE
Qwen3.6 is the latest generation of the Qwen family and one of the strongest open-weight model lines available today, spanning chat, reasoning, tool use, and agentic workflows. It is an MoE model that delivers the learned capacity of a 35B network at roughly the decoding cost of a 3B one.
An MoE model has multiple specialized sub-networks (experts) inside each layer. For each generated token, a learned router selects only a few of those experts to run, rather than the full model. Reading only a fraction of the weights at each step keeps latency and memory bandwidth in check, making it a strong fit for on-device AI PCs and a stepping stone toward local AI.
The challenge: accelerating MoE models with SD
SD speeds up generation by having a small, fast draft model propose several tokens, while a larger target model verifies them in a single pass. If several draft tokens are accepted, generation advances by multiple tokens for almost the same cost as producing a single token with standard autoregressive decoding (see the original paper).
That cost assumption is much weaker for MoE models. In a dense model, each target-model pass touches essentially the same weights whether it decodes one token or verifies a short sequence of drafted tokens. In an MoE model, each token can route to different experts, so verifying several drafted tokens may require loading far more experts than a single-token decoding step.
This is why Qwen3.6-35B-A3B is a demanding test case: it combines MoE routing with 4-bit weight quantization, a state-of-the-art inference setup that already reduces memory traffic. SD must therefore deliver gains on top of an already efficient baseline.
DFlash on Intel® Panther Lake
DFlash is a speculative decoding method that replaces the usual autoregressive draft model with a lightweight parallel masked-token drafter. Instead of proposing draft tokens one by one, it proposes a short block in parallel. Similar to EAGLE-style drafters, DFlash conditions its block proposal on hidden states produced by the target model, so the drafter predicts from the target model's internal representation of the prefix rather than from token IDs alone.
We implemented a new DFlash speculative decoding pipeline in OpenVINO.GenAI, and also added DFlash export support to HuggingFace's optimum-intel. For our experiments we used Qwen/Qwen3.6-35B-A3B and z-lab/Qwen3.6-35B-A3B-DFlash both quantized to int4 (W4A16). Quantization was done with OpenVINO'S NNCF. We then benchmarked the new DFlash pipeline on three datasets commonly used for speculative decoding benchmarking: HumanEval (coding), MT-Bench (assistant) and GSM8K (math).
Results
We measured throughput and average acceptance length (how many tokens are produced per step) across HumanEval, MT-Bench and GSM8K datasets. We set the number of assistant tokens to 7. The baseline throughput for Qwen3.6-35B-A3B W4A16 is 41 t/s, which is the current state-of-the-art, and the table below reports the speedups we measured on top of that.
| Benchmark | Throughput (tokens/s) | Accept. length (tokens) | Speedup |
|---|---|---|---|
| HumanEval | 89.8 | 6.4 | 2.2× |
| MT-Bench | 54.7 | 4.0 | 1.3× |
| GSM8K | 68.5 | 5.0 | 1.6× |
Throughput measured on the Intel® Panther Lake iGPU. Speedup is DFlash throughput relative to the target baseline.
The acceptance length, and therefore the speedup, is higher for HumanEval and GSM8K than for MT-Bench. This is consistent with the DFlash paper: structured, predictable outputs like code and step-by-step math let the drafter correctly guess longer runs of tokens, while open-ended chat is harder to predict.
Other Qwen deployment points
While Qwen3.6-35B-A3B is the main focus of this post, the same DFlash pipeline also enables SD for dense Qwen models such as Qwen3.6-27B and Qwen3.5-9B. These models target different local-AI tradeoffs.
| Model | Benchmark | Throughput (tokens/s) | Accept. length (tokens) | Speedup |
|---|---|---|---|---|
| Qwen3.6-27B | HumanEval | 38.3 | 6.9 | 5.5x |
| MT-Bench | 22.1 | 4.2 | 3.1x | |
| GSM8K | 28.4 | 5.2 | 4.0x | |
| Qwen3.5-9B | HumanEval | 76.0 | 5.4 | 3.3x |
| MT-Bench | 59.1 | 4.3 | 2.8x | |
| GSM8K | 70.6 | 5.0 | 3.4x |
Qwen3.5-9B has a much smaller memory footprint and faster TTFT while keeping decoding speed close to the 35B-A3B setup, at the cost of lower quality. Qwen3.6-27B goes the other way: its memory footprint and TTFT are closer to the 35B-A3B model, and decoding is slower, but its quality is substantially higher and competitive with much larger models.
Together, these results show that DFlash is useful not only for the 35B-A3B MoE case, but also gives developers a practical way to balance footprint, latency, throughput and quality across the Qwen family.
Conclusion
MoE models are designed to be efficient, which makes them a hard target for further optimization. With DFlash and OpenVINO on Intel® Core™ Ultra (Panther Lake), we show more than 2× faster generation for Qwen3.6-35B-A3B-int4 on code, 1.6x speedup on Math and a solid 1.3× on chat. Paired with DFlash and OpenVINO, Qwen3.6-35B-A3B offers one of the strongest quality-to-latency tradeoffs among open-weight models. In addition, Qwen3.6-27B and Qwen3.5-9B provide two more local-AI entry points, allowing developers to choose the right balance of quality, latency, and memory footprint for their workload. Together, these results advance practical, high-quality local AI on AI PCs by extending DFlash acceleration across both MoE and dense Qwen models.
the DFlash pipeline currently supports text-only inputs, a single request at a time, and greedy sampling only, and does not yet support prefix caching - these capabilities are planned for upcoming OpenVINO.GenAI releases.
Performance and legal notices
Performance results are based on internal benchmarking with OpenVINO™ 2026.3.0-22113-877cbcadc2c as of 24 June, 2026, using an Intel® Core™ Ultra (Panther Lake) Intel Core Ultra X7 368H laptop with GPU and 64 GB of memory. Run on Intel Arc B390 iGPU with driver 32.0.101.8860
Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.
No product or component can be absolutely secure.
Your costs and results may vary.
Intel technologies may require enabled hardware, software, or service activation.
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.
