Instructions to use Qwen/Qwen3.8-Flash-Next with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Qwen/Qwen3.8-Flash-Next with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Qwen/Qwen3.8-Flash-Next")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Qwen/Qwen3.8-Flash-Next")
model = AutoModelForMultimodalLM.from_pretrained("Qwen/Qwen3.8-Flash-Next", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Qwen/Qwen3.8-Flash-Next with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Qwen/Qwen3.8-Flash-Next"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3.8-Flash-Next",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Qwen/Qwen3.8-Flash-Next

SGLang

How to use Qwen/Qwen3.8-Flash-Next with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Qwen/Qwen3.8-Flash-Next" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3.8-Flash-Next",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Qwen/Qwen3.8-Flash-Next" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3.8-Flash-Next",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Qwen/Qwen3.8-Flash-Next with Docker Model Runner:
```
docker model run hf.co/Qwen/Qwen3.8-Flash-Next
```

can you please make a 39b dense model?

#23

by Manni1000 - opened 7 days ago

Discussion

Manni1000

7 days ago

can you please make a 39b dense model? i think that would be the biggest size that could fit in the 32gb from a 5090. when such a model fully fits into the gpu with no offloading is is still very fast even thogh its dense. it could be a peak coding model with a small footprint. i think it would be quite interesting.

hgeist

7 days ago

If you have a 32gb 5090 and 60gb ram, you can run unsloth ud q4_xs on that setup and qwen3.8 flash next delivers more intelligence than qwen3.8 27b already.

This way you can even steer your kv cache size / context size better than if you had a dense model.

After some optimizations and when unsloth got mtp in their q4 fixed, you will have much higher speeds than any dense model. It's perfect.

shing3232

7 days ago

No, because it would be much expensive compare to this

MrDevolver

7 days ago

•

edited 7 days ago

If you have a 32gb 5090 and 60gb ram, you can run unsloth ud q4_xs on that setup and qwen3.8 flash next delivers more intelligence than qwen3.8 27b already.

This way you can even steer your kv cache size / context size better than if you had a dense model.

After some optimizations and when unsloth got mtp in their q4 fixed, you will have much higher speeds than any dense model. It's perfect.

I have a borrowed laptop with 64GB RAM LPDDR5X, Nvidia RTX 5090 24GB GPU. Your suggested setup has 32GB GPU, but iq4_xs quant which means that stepping into lower area of quants should still work, but it does not. Bionic agent (same creators who created LM Studio) says that even the smallest quant is too big for this setup.

CoderX1997

6 days ago

Theoretically - a 32gb 5090 is way more capable then u think. If you were to use 1 bit quants (impractical) you could get 8 paramaters / byte. Meaning you could squeeze 256B onto the 5090. (32GB * 8)

Or if you offloaded to disk, you could get a lot more.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment