Instructions to use barozp/Qwen3.8-27B-Opus-Distill with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use barozp/Qwen3.8-27B-Opus-Distill with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="barozp/Qwen3.8-27B-Opus-Distill")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("barozp/Qwen3.8-27B-Opus-Distill")
model = AutoModelForMultimodalLM.from_pretrained("barozp/Qwen3.8-27B-Opus-Distill", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use barozp/Qwen3.8-27B-Opus-Distill with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "barozp/Qwen3.8-27B-Opus-Distill"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "barozp/Qwen3.8-27B-Opus-Distill",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/barozp/Qwen3.8-27B-Opus-Distill

SGLang

How to use barozp/Qwen3.8-27B-Opus-Distill with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "barozp/Qwen3.8-27B-Opus-Distill" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "barozp/Qwen3.8-27B-Opus-Distill",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "barozp/Qwen3.8-27B-Opus-Distill" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "barozp/Qwen3.8-27B-Opus-Distill",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use barozp/Qwen3.8-27B-Opus-Distill with Docker Model Runner:
```
docker model run hf.co/barozp/Qwen3.8-27B-Opus-Distill
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Qwen3.8-27B-Opus-Distill

Qwen/Qwen3.8-27B fine-tuned (LoRA, merged) on Claude Opus reasoning traces. Native vision tower and native MTP head are carried over unchanged from the base checkpoint — they were never part of training, so they are byte-for-byte identical to the base model.

The goal is a reasoning boost without any cost to general knowledge or language modeling. Measured: reasoning improves (ARC +4.2pt, GPQA +26pt), knowledge and language modeling stay flat. See the benchmark table below.

Known issues

Reasoning loop under stacked output-format constraints. Reported by zxbc2023 (full writeup, discussion #1). Combining "no prose" with a second output-format constraint (e.g. "no markdown" or "no comments") can send this model into a non-converging self-verification reasoning loop -- it burns the entire token budget with zero visible output. Fully deterministic and reproducible at temp=0. Root cause: traced to part of the training data being sourced from reconstructed (not verbatim) Opus reasoning traces, not a capability gap.

Fixed in barozp/Qwen3.8-27B-Opus-Distill-v2 -- retrained on a rebuilt dataset where every row is traced to a verified genuine source. If you're hitting this, switch to v2.

Workaround if staying on this version: avoid combining "no prose" with another format constraint, or raise the generation token budget to >=4096 for constrained code-gen tasks.

Model overview

Base model: Qwen/Qwen3.8-27B — dense 27B, native vision-language, native MTP.
Method: LoRA (r=64, alpha=64, dropout=0.05), merged into the base weights.
LoRA targets: attention q/k/v/o_proj on the 16 full-attention layers only; FFN gate/up/down_proj on all 64 layers. Gated-DeltaNet (linear-attention) projections are left untouched, matching the Qwen3.6 recipe.
Training data: barozp/opus-reasoning-distill-train (14,250 examples) + -validation (750, held out).
Training run: 1 epoch (891 steps), lr=1e-4 cosine with 3% warmup, effective batch 16 (micro-batch 1 × grad-accum 16), MAX_SEQ=4096, bf16, ~5h52m on an A100 80GB.
Final validation loss: 0.4647 (step 891/891).
Vision + MTP: carried over from base, never trained.

Highlights

Reasoning improves, nothing else degrades. ARC-Challenge and GPQA-Diamond both rise while MMLU (general knowledge) and wikitext (language modeling) stay flat within noise.
Reflex-level reasoning gets the biggest boost. GPQA is measured in loglikelihood mode with thinking disabled — the model has no chance to deliberate. Distillation more than doubles that "reflex" score, meaning the model internalized step-by-step reasoning rather than just learning to generate <think> blocks.
Vision and MTP are fully intact. This is a complete multimodal model with self-speculative decoding, not a text-only strip-down.

Benchmark results

Measured with lm-evaluation-harness, 0-shot, loglikelihood (multiple-choice), chat template OFF, QUICK mode (--limit 500). Base and distill were run with the identical harness, so the Δ column is the meaningful signal — not the absolute values.

Task	Metric	Base	Distill	Δ
wikitext	word perplexity ↓	8.434	8.344	−0.09
mmlu	acc	0.849	0.849	−0.001
hellaswag	acc_norm	0.742	0.740	−0.002
arc_challenge	acc_norm	0.588	0.630	+0.042
gpqa_diamond	acc_norm	0.232	0.495	+0.263

Important caveats

GPQA is not comparable to Qwen's published 89.2. Qwen reports GPQA with thinking mode ON and their own harness. Here GPQA is measured in loglikelihood mode with thinking OFF — the base model scores near random (25%) precisely because it gets no chance to reason. The +26pt Δ is a valid same-protocol comparison (base vs distill), but do not place 0.495 next to Qwen's 89.2 as if they were the same measurement.
ARC-Challenge is saturated for modern models. The +4.2pt Δ is real and consistent with the Qwen3.6 release, but ARC (2018, middle-school science) is not a headline reasoning benchmark anymore. GPQA is the stronger signal here.
QUICK mode limits each task to 500 samples; hellaswag and arc_challenge are noisy at that size. Treat small Δ values (±0.01) on those as noise.

Quick start

from transformers import AutoModelForImageTextToText, AutoProcessor
import torch

model = AutoModelForImageTextToText.from_pretrained(
    "barozp/Qwen3.8-27B-Opus-Distill", dtype=torch.bfloat16, device_map="auto",
)
processor = AutoProcessor.from_pretrained("barozp/Qwen3.8-27B-Opus-Distill")

Text-only inference also works via AutoModelForCausalLM (vision is ignored, which is the expected/standard behavior for that class).

Related models

Qwen3.8-27B — the base model.
barozp/Qwen3.8-27B-Opus-Distill-LoRA-Adapter — the raw LoRA adapter (private; the merged weights here are the same thing applied).
barozp/Qwen3.8-27B-Opus-Distill-GGUF — GGUF quantizations, BF16 down to IQ1_M, imatrix-calibrated, includes the vision mmproj file.
barozp/Qwen3.8-27B-Opus-Distill-v2 — bugfix release, see Known Issues above.