Instructions to use Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked")
model = AutoModelForMultimodalLM.from_pretrained("Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked

SGLang

How to use Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked with Docker Model Runner:
```
docker model run hf.co/Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked
```

Qwopus3.8-27B-Coder-Flash, GPTQ INT4 baked (v1.1.1)

Jackrong/Qwopus3.8-27B-Coder-Flash quantized from BF16 to INT4 by Launch80. The lm_head and the MTP draft head are INT4 on disk too, so a single Intel Arc Pro B65 32 GB serves it at the full 262,144-token context, and stock vLLM loads it with no weight patches.

This model is a work in progress, and the full story is public:

The study, with every number, the bug we found and how we fixed it: https://launch80.com/blog/qwopus-coder-int4-bake-study
Follow along and ask questions in the Launch80 Discord: https://discord.gg/launch80

version	what changed	status
v1.0 (tag `v1.0`)	GPTQ INT4 body calibrated with tool-call data, baked INT4 head and MTP	released 2 Nov 2026
v1.0.1 (tag `v1.0.1`)	card only, weights unchanged: all seven tasks measured on this exact checkpoint, served as shipped (MTP k=6)	released 2 Nov 2026
v1.1 (tag `v1.1`)	weights unchanged; adds a fast-decode serving profile with our INT4 DFlash2 drafter (k=7, 128k): 89.3 tok/s on the B65	released 3 Nov 2026
v1.1.1 (this revision, tag `v1.1.1`)	card only, weights unchanged: one more vLLM-XPU boot patch removes a per-step host sync in the verify step; fast profile 92.5 / 96.0 tok/s (two quick runs)	released 3 Nov 2026

What's in it

tensor	source (BF16)	this repo
400 body linears	BF16	GPTQ INT4 sym g128 (GPTQModel 7.3.2)
`lm_head` (248,320 × 5,120)	BF16, 2.54 GB	GPTQ INT4 sym g128, Hessian-calibrated
`mtp.*` draft: `fc`, q/k/v/o, gate/up/down	BF16	GPTQ INT4 sym g128, Hessian-calibrated
embeddings, norms, vision tower, GDN `in_proj_a`/`in_proj_b`	BF16	unchanged (byte-identical)

17.0 GB on disk. The tokenizer, chat template and processor files are the source repo's, unchanged.

Body. GPTQModel 7.3.2 ran on the Arc Pro B65 itself with these settings:

4-bit, symmetric, group 128
desc_act off, act_group_aware on
damping 0.05, seed 608
512 × 2048-token calibration samples: 153 C4, 204 code, 53 chat and 102 tool-call conversations. The tool-call samples are from NousResearch/hermes-function-calling-v1 @dae3e1d, rendered through the model's own chat template with tools=, half of them parallel calls.

The calibration sha256 is in quant-meta.json.

Head and MTP bake. These steps follow the layout of our earlier bakes:

Forward hooks inside vLLM captured Hessians on wikitext, chat and code slices.
Each of the nine linears was GPTQ-quantized under a grid of dampings and slice mixes.
The setting with the lowest output error on held-out rows was kept.

Every baked linear beats round-to-nearest (RTN):

linear	baked	RTN
`lm_head`	4.68%	8.18%
`mtp.fc`	6.98%	11.47%
`mtp…q_proj`	2.68%	4.44%
`mtp…k_proj`	6.05%	10.08%
`mtp…v_proj`	5.11%	8.28%
`mtp…o_proj`	6.67%	13.04%
`mtp…gate_proj`	3.67%	6.35%
`mtp…up_proj`	5.70%	9.56%
`mtp…down_proj`	7.57%	11.96%

Quality vs BF16 (pre-registered)

We wrote the thresholds down and hashed them before any quantized model was evaluated. The BF16 reference ran on a rented RTX PRO 6000 (vLLM 0.30, CUDA); this checkpoint ran on the Arc Pro B65 (vLLM 0.30, XPU), served exactly as shipped, with MTP k=6 on for every task and for the logprobs. Full study: https://launch80.com/a/84829a3a-a0b2-407f-ab38-59cd2d5006fa

#	measure	bar	this checkpoint
T1	wikitext-2 perplexity	≤ +3%	+1.72%	pass
T2	code-corpus perplexity	≤ +3%	+1.83%	pass
T3	mean next-token KL divergence (1.05M tokens, top-64)	≤ 0.05	0.0412	pass
T4	top-1 next-token agreement	≥ 93%	92.84%	fail (by 0.16 pt)
T5	seven tasks, paired pass@1, greedy	Δ ≥ −2 pts or CI includes 0	7/7 tasks (table below)	pass
T6	KLD vs Jackrong's Q4_K_M GGUF	ours ≤ theirs	0.0412 vs 0.0250	fail
T7	greedy, spec decoding on = off	identical	26/32; all 6 differences are at a tie (3 exact, 3 a 0.0156 FP16 step)	fail as written

task	BF16	this checkpoint	Δ pts [95% CI]
GSM8K	86.96	86.66	−0.30 [−1.21, +0.61]
IFEval	80.22	80.78	+0.55 [−2.22, +3.14]
MMLU-Pro subset	71.80	70.90	−0.90 [−2.70, +0.90]
HumanEval+	90.24	88.41	−1.83 [−5.49, +1.83]
MBPP+	76.98	75.66	−1.32 [−3.17, +0.53]
BFCL subset (400)	89.00	88.25	−0.75 [−3.00, +1.50]
LiveCodeBench v6 slice	41.14	40.57	−0.57 [−5.14, +4.00]

T6, against Jackrong's Q4_K_M GGUF: fail. On the same 512 × 2048 tokens, Q4_K_M stays closer to BF16: mean KLD 0.0250 vs our 0.0412, top-1 agreement 94.92% vs 92.84%, perplexity +0.73% vs +1.72% (wikitext-2). The files are about the same size (16.8 vs 17.1 GB); Q4_K_M keeps 67 tensors at 6 bits, while this checkpoint carries a BF16 vision tower and fp16 embeddings and a uniform 4-bit g128 body. Method: the GGUF ran in llama.cpp (b11723, SYCL) on the B65, its top-64 logprobs were scored with the same code against the same BF16 reference. This differs from the registered method (a BF16 GGUF base, which needed a rental) and is dated in the thresholds file. Q8_0 measured the same way gives 0.0024, so the cross-engine term is small. Pick Q4_K_M for fidelity on llama.cpp; pick this build for vLLM with speculative decoding.

Needle retrieval: 100% at 32k and at 128k. MTP draft acceptance over the whole suite: 46.8% of drafted tokens (80.2 / 62.2 / 48.0 / 37.4 / 29.5 / 23.3% by draft position), about 3.8 tokens per verify step under greedy decoding.

Disclosure: the tool-call calibration is a post-hoc fix. The first bake, with no tool calls in its calibration set, passed six of the seven tasks but lost 7.75 points on BFCL. It ended its turn after the first of several parallel calls. We traced that to the calibration data, recalibrated the body, and rebaked the head and MTP. This checkpoint is that rebake, and the table above measures it on all seven tasks. The change was made after seeing results, so treat it as post hoc. The blog post has the whole investigation.

Speed (Arc Pro B65 32 GB, vLLM 0.30.0 XPU, fp8 KV, single stream)

Two serving profiles, same weights. Both are quick BetterBench runs (1 warmup + 5 passes per category):

profile	drafter	context	weighted decode	page
fast decode + GDN no-sync patch (v1.1.1)	Launch80/Qwen3.8-27B-DFlash2-Int4-baked-drafter, k=7	131,072	92.5 / 96.0 tok/s (two runs)	https://launch80.com/a/81dea056-a018-4e48-9f5e-b89b039f368d
fast decode, v1.1 (no patch)	same	131,072	89.3 tok/s	https://launch80.com/a/54608011-fd4d-4c68-9d32-868a2fcdd03b
full context	built-in MTP head, k=6	262,144	75.8 tok/s	https://launch80.com/a/30aa8ae2-f00c-409a-a4a9-2e298fc81f67

The fast profile gains most on structured output (5.2 accepted tokens per step on JSON, 4.7 on summaries) and least on long thinking and prose (2.8 and 2.5). It costs context: with the drafter loaded, the 32 GB card fits 128k, not 262k. Spec-decoding parity (T7) for the fast profile with the v1.1.1 patch: 30 of 32 greedy outputs identical to spec-off, and both differences are at exact top-2 ties (v1.1 without the patch: 28 of 32, all ties), the same pattern as the MTP profile.

What v1.1.1 fixes: on every speculative step, vLLM's GDN (linear-attention) metadata builder indexed GPU tensors with a CPU boolean mask. That forces the host to wait for the GPU, which cancelled async scheduling and left ~7 ms of host time in each step at k=7. patch_gdn_spec_nosync.py replaces the mask with the equivalent slice: 47.9 → 44.4 ms per step on a single prompt. We also tried a Triton INT4 GEMM for the 9-to-16-row verify shapes; it ran 3.5× slower than oneDNN, so the fast profile stays at k=7. Quick BetterBench runs vary by a few tok/s between runs, so both runs are shown.

The MTP head is byte-identical to base Qwen3.8-27B (the fine-tune never trained it). We also fine-tuned the DFlash2 drafter on 10,000 of this model's own answers; no checkpoint beat the stock drafter, so v1.1 uses the stock drafter, INT4-baked. Drafter study: https://launch80.com/a/a7b8a95a-e2c5-473c-b017-310232fc56e0

Serving

Full-context profile (75.8 tok/s, 262k context):

vllm serve Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked \
  --quantization gptq --dtype float16 --kv-cache-dtype fp8 --max-model-len 262144 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":6}' \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml --reasoning-parser qwen3

Fast-decode profile (92.5 / 96.0 tok/s, 128k context):

vllm serve Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked \
  --quantization gptq --dtype float16 --kv-cache-dtype fp8 --max-model-len 131072 \
  --speculative-config '{"method":"dflash","model":"Launch80/Qwen3.8-27B-DFlash2-Int4-baked-drafter","num_speculative_tokens":7,"quantization":"gptq"}' \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml --reasoning-parser qwen3

The drafter must run in bf16 while this target runs in fp16: on XPU that needs the patch_dflash_mixed_dtype.py boot patch from launch80/B65 (on CUDA, serve both in bf16 with --dtype bfloat16).

Every number above was measured with these configurations (vLLM 0.30.0 XPU image), plus three vLLM-XPU correctness patches for speculative decoding (MTP nightly, MTP boundary, GDN mixed-split) from the same repository. The fast profile's v1.1.1 number also needs patch_gdn_spec_nosync.py (run it after the MTP boundary patch); without it the same command gives 89.3. Tool calling needs qwen3_xml, not hermes. Pin a revision (--revision v1.1.1) if you want this exact build; main moves as new versions ship.

License

Apache-2.0, as the source model and the Qwen3.8-27B base are.

Credits

Qwen (Qwen3.8-27B) · Unsloth (the base Jackrong built on) · Jackrong (Qwopus3.8-27B-Coder-Flash) · NousResearch (hermes-function-calling-v1, used for calibration) · GPTQModel · Inco AI / z-lab (the DFlash2 drafter) · SpecForge (the drafter fine-tune experiment) · Wrapzii (asked for this bake, and the on-disk bake layout it follows).