Instructions to use inference-optimization/Qwen3.6-1.3B-A1.3B-small with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use inference-optimization/Qwen3.6-1.3B-A1.3B-small with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="inference-optimization/Qwen3.6-1.3B-A1.3B-small")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("inference-optimization/Qwen3.6-1.3B-A1.3B-small")
model = AutoModelForMultimodalLM.from_pretrained("inference-optimization/Qwen3.6-1.3B-A1.3B-small", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use inference-optimization/Qwen3.6-1.3B-A1.3B-small with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "inference-optimization/Qwen3.6-1.3B-A1.3B-small"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inference-optimization/Qwen3.6-1.3B-A1.3B-small",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/inference-optimization/Qwen3.6-1.3B-A1.3B-small

SGLang

How to use inference-optimization/Qwen3.6-1.3B-A1.3B-small with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "inference-optimization/Qwen3.6-1.3B-A1.3B-small" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inference-optimization/Qwen3.6-1.3B-A1.3B-small",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "inference-optimization/Qwen3.6-1.3B-A1.3B-small" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inference-optimization/Qwen3.6-1.3B-A1.3B-small",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use inference-optimization/Qwen3.6-1.3B-A1.3B-small with Docker Model Runner:
```
docker model run hf.co/inference-optimization/Qwen3.6-1.3B-A1.3B-small
```

Qwen3.6-1.3B-A1.3B-small

This is a small version of Qwen/Qwen3.6-35B-A3B created for testing and development. It keeps the base model's hidden dimensions and all hyperparameters identical, reducing only the layer counts (text + vision towers), the per-layer type list, and the number of MoE experts, so that it exercises the same architecture (multimodal Qwen3.5-MoE VLM with hybrid linear-attention / full-attention text tower) in vLLM and transformers.

Model Details

Base Model: Qwen/Qwen3.6-35B-A3B
Architecture: qwen3_5_moe (Qwen3_5MoeForConditionalGeneration, multimodal)
Total Parameters: 1.32B (1,324,459,616)
Activated Parameters: 1.26B (1,258,875,968; per-token text-model active params, vision tower excluded — with 8 experts and top-k 8 all experts are active per token)

Configuration Changes

Only the following config fields were changed relative to the base model (all other hyperparameters — text hidden size 2048, vocab 248320, moe/shared intermediate sizes 512, num_experts_per_tok 8, mrope (interleaved, section [11,11,10]), rope theta, head dims, linear-attention dims, vision hidden size 1152 / patch 16 / merger, tie_word_embeddings: false, etc. — are identical):

Parameter	Base (Qwen3.6-35B-A3B)	Small
`text_config.num_hidden_layers`	40	4
`text_config.layer_types`	30x `linear_attention` + 10x `full_attention` (3:1 repeating)	3x `linear_attention` + 1x `full_attention`
`text_config.num_experts`	256	8
`vision_config.depth`	27	2

The 4-layer text tower is exactly one repeating unit of the base's 3:1 linear/full attention pattern, covering every distinct attention layer type; every text layer is MoE (the base has no dense MLP layers). The expert count (8) equals the base top-k (8). The untied 248320x2048 input/output embeddings (~1.02B params) dominate the parameter count, so ~1.3B is near the minimum for this architecture with fixed dims.

Checkpoint Structure

Single model.safetensors file (bf16, ~2.6 GB). Tensor naming follows the base checkpoint exactly (model.language_model.layers.{i}.linear_attn.*, model.language_model.layers.{i}.self_attn.*, model.language_model.layers.{i}.mlp.experts.gate_up_proj/down_proj grouped expert tensors, model.language_model.layers.{i}.mlp.shared_expert.*, model.visual.blocks.{i}.*, model.visual.merger.*, model.visual.patch_embed.*, model.visual.pos_embed.weight, lm_head.weight), verified against the base model.safetensors.index.json — all 48 tensor-name patterns match modulo layer count. MTP layers of the base (mtp.*) are intentionally not materialized (see Notes). Processor files (processor_config.json, preprocessor_config.json, video_preprocessor_config.json, tokenizer files, chat template) are included.

Usage

from transformers import AutoModelForImageTextToText, AutoProcessor

model = AutoModelForImageTextToText.from_pretrained("inference-optimization/Qwen3.6-1.3B-A1.3B-small", device_map="auto")
processor = AutoProcessor.from_pretrained("inference-optimization/Qwen3.6-1.3B-A1.3B-small")

input_ids = processor.tokenizer("According to all known laws", return_tensors="pt").input_ids.to(model.device)
output = model.generate(input_ids, max_new_tokens=20)
print(processor.tokenizer.decode(output[0]))

Creation Process

This model was created using the llm-compressor create-tiny-model skill.

The base AutoConfig was loaded and only the allowed changes above were applied.
The model was built without downloading base weights, weights were initialized (init_weights + NaN/extreme-value fixup), and saved together with the base's processor.
The model was fine-tuned on a small text-only copypasta dataset until perplexity reached ~1.0 (well below the 3.0 target; training loss ~0.018).
Checkpoint structure was validated against the base model.safetensors.index.json, and the multimodal (image) forward path was smoke-tested through the processor and vision tower.

Validation

$ python validate_tiny_model.py inference-optimization/Qwen3.6-1.3B-A1.3B-small
Success: 1.0002583265304565 <= 10.0

==================================================
Generating sample text:
According to all known laws of aviation, there is no way a bee should be able to fly.
==================================================

Multimodal smoke test (random image through processor + vision tower): forward pass produces finite logits of shape (1, 80, 248320); generation runs end-to-end.

Notes

mtp_num_hidden_layers: 1 and mtp_use_dedicated_embeddings: false are kept from the base config. As with other models of this family, transformers materializes a backbone-only checkpoint by design: no mtp.* weights are included, matching the convention of other entries in this collection.
The base is unquantized bf16; no quantization_config was present or added.
The model was fine-tuned on text only (per the creation workflow); vision-tower weights retain their random initialization, so image inputs produce arbitrary (but numerically valid) outputs.
Zero-initialized RMSNorm weights ((1 + weight) parameterization) are the intended init scheme of this architecture.

Downloads last month: 627

Safetensors

Model size

1B params

Tensor type

BF16

Model tree for inference-optimization/Qwen3.6-1.3B-A1.3B-small

Base model

Qwen/Qwen3.6-35B-A3B

Finetuned

(399)

this model

Collection including inference-optimization/Qwen3.6-1.3B-A1.3B-small

Small Models

Collection

Small versions of known models: same vLLM loading characteristics as the base, fewer layers/experts, identical hidden dims and hyperparameters • 25 items • Updated 6 days ago