Qwen3.8 Distilled 9B Now With Vision (update with flm-add --force)

FastFlowLM Q4NX conversion of empero-ai/Qwen3.8-9B for AMD XDNA NPU inference.

This repository contains a quantized Q4NX port of the model, compiled for the FastFlowLM (FLM) runtime. It is not a GGUF file.

Item	Value
Source model	`empero-ai/Qwen3.8-9B`
Source GGUF	`Qwen3.8-9B-Q8_0.gguf`
Weights	`model.q4nx` (7.23 GB)
Modality	language
FLM version	`1.0.1`
Converted	2026-08-16

Source repository

Metadata from the upstream Hugging Face repository:

Item	Value
License	apache-2.0
Base model	`Qwen/Qwen3.5-9B`
Library	`transformers`
Model type	`qwen3_5`
Pipeline	`text-generation`
Downloads	128
Repo revision	`0934f3d2327ff2df2197495278c4c46ae5a56bd9`

Files

File	Description
`model.q4nx`	Quantized weights (Q8_0 / Q4_1 / BF16)
`config.json`	FLM runtime configuration
`tokenizer.json`	Tokenizer vocabulary
`tokenizer_config.json`	Tokenizer configuration
`chat_template.jinja`	Chat template
`vision_weight.q4nx`	Vision model

Install and run

This repository works with flm-add, a small installer that copies the model into the FastFlowLM user directory and registers the tag. It never modifies the system FastFlowLM install.

pip install flm-add or uv tool install flm-add

uv tool install flm-add
flm-add Atomic-Germ/Qwen3.8-Distilled-9B-NPU2 --tag qwen3.8-distilled:9b --family qwen3.5 --xclbin-from Qwen3.5-9B-NPU2 (with --force to update)
FLM_CONFIG_PATH="$HOME/.config/flm/model_list.json" FLM_XCLBIN_PATH="$HOME/.config/flm" flm run qwen3.8-distilled:9b

[FLM] === Benchmark Results ===

Context Length	TTFT (s)	Prefill Speed (tok/s)	Decoding Speed (tok/s)
1k	5.696 ± 0.011	171.65 ± 0.31	5.96 ± 0.00
2k	8.865 ± 0.036	219.42 ± 0.90	5.91 ± 0.00
4k	15.173 ± 0.007	255.69 ± 0.12	5.83 ± 0.00
8k	28.214 ± 0.007	274.63 ± 0.07	5.66 ± 0.00
16k	54.699 ± 0.130	283.12 ± 0.67	5.36 ± 0.00
32k	112.713 ± 1.332	274.75 ± 3.24	4.80 ± 0.04

Source model card

Qwen3.8-9B

Developed by Empero

This repository contains model weights and configuration files in the Hugging Face Transformers format.

These artifacts are compatible with Hugging Face Transformers, vLLM, SGLang, and other standard runtimes with Qwen3.5 architecture support.

Qwen3.8-9B is a full-parameter distillation of Qwen3.8 2.4T A95B into the Qwen3.5-9B architecture. The student was trained on ~70,000 curated teacher traces from our internal Qwen3.8 distillation datasets — dense chain-of-thought spanning mathematics, code, general reasoning, instruction following, and tool use, quality-filtered before training.

The objective: bring the reasoning behavior of a frontier-scale teacher into a dense 9B that deploys on a single GPU.

Highlights

Distilled chain-of-thought — every answer opens with a <think> block learned directly from Qwen3.8 2.4T A95B traces rather than synthetic self-generated reasoning.
Mathematics and code emphasis — the trace mix is deliberately weighted toward hard math and competitive programming, the domains where distillation moves the needle most at this scale.
Native function calling per Qwen3.5's specification — no wrapper or tool-specific fine-tune required.
262,144-token native context, inherited from the Qwen3.5 base.
Full fine-tune — every parameter updated; not an adapter.