Instructions to use irl-kit/SPARC-Qwen3.5-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use irl-kit/SPARC-Qwen3.5-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="irl-kit/SPARC-Qwen3.5-4B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("irl-kit/SPARC-Qwen3.5-4B")
model = AutoModelForMultimodalLM.from_pretrained("irl-kit/SPARC-Qwen3.5-4B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use irl-kit/SPARC-Qwen3.5-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "irl-kit/SPARC-Qwen3.5-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "irl-kit/SPARC-Qwen3.5-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/irl-kit/SPARC-Qwen3.5-4B

SGLang

How to use irl-kit/SPARC-Qwen3.5-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "irl-kit/SPARC-Qwen3.5-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "irl-kit/SPARC-Qwen3.5-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "irl-kit/SPARC-Qwen3.5-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "irl-kit/SPARC-Qwen3.5-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use irl-kit/SPARC-Qwen3.5-4B with Docker Model Runner:
```
docker model run hf.co/irl-kit/SPARC-Qwen3.5-4B
```

SPARC-Qwen3.5-4B / README.md

holgerson

Update README.md

7d86ac4 verified 6 days ago

preview code

Raw

History Blame Contribute Delete

3.02 kB

metadata

library_name: transformers
pipeline_tag: image-text-to-text
tags:
  - robotics
  - vision-language
  - spatial-reasoning
  - sparc
datasets:
  - irl-kit/SPARC-VQA

SPARC-Qwen3.5-4B

Qwen3.5-4B fully fine-tuned for embodied spatial reasoning using VQA data generated from SPARC annotations.

Training data

The training mixture contains SPARC-generated VQA data from ours_adaptive_det_soft_snr_sp8, FSD, RoboPoint, and LLaVA-OneVision2. SPARC samples use an annotation-quality threshold of 0.97, are sorted by score, and are capped at 700 samples per object. The paper reports 1,159,047 training pairs for this mixture.

Release	SPARC VQA (filtered)	FSD	RoboPoint	LLaVA-OneVision2	EO-1.5M
Qwen3.5-4B	Yes	Yes	Yes	Yes	No
Qwen3.5-0.8B-VTFT	Yes	Yes	Yes	Yes	No
Qwen3.5-9B-EO	Yes	Yes	Yes	Yes	Yes

The SPARC training data is the filtered release subset, which needs no further SPARC filtering. Its filtering script and release mixture manifest are in irl-kit/SPARC-VQA-Raw. FSD, RoboPoint, LLaVA-OneVision2, and EO-1.5M remain their respective upstream datasets.

Prompting

Prompt formatting is important for these models. Use the bundled chat_template.jinja through processor.apply_chat_template(..., add_generation_prompt=True), with one user turn containing the image(s) followed by the text question. Disable thinking/reasoning mode to match evaluation.

For a single point, append exactly:

Output the point coordinates in JSON format like [{"point_2d": [x, y], "label": "target"}]. Use integer coordinates between 0 and 1000.

For a trajectory or multiple points, append exactly:

Return only a JSON list like [{"point_2d": [x1, y1], "label": "point_1"}, {"point_2d": [x2, y2], "label": "point_2"}, ...]. Use integer coordinates between 0 and 1000.

Training

The vision encoder is frozen and the vision projector is trainable. The model was fully fine-tuned for one epoch with a learning rate of 2e-5 and a maximum sequence length of 5600.

Evaluation

This is the paper's primary 4B model. The paper reports a 62.7 pointing/VQA average for this model. The local full benchmark evaluation associates the released weights with aggregate score 0.698.

Model	Aggregate	Where2Place	RefSpatial location	IA-Bench	RoboRefIt testA	VA Bench-P
Qwen3.5-4B	0.698	72.0	59.0	79.0	85.7	65.7
Qwen3.5-0.8B-VTFT	0.605	58.0	47.0	76.7	80.9	48.3
Qwen3.5-9B-EO	0.719	76.0	68.0	78.5	85.2	68.7

Citation

@article{blank2026sparc,
  title={SPARC: Reliable Spatial Annotations from Robot Demonstrations at Scale},
  author={Blank, Nils and others},
  journal={arXiv preprint arXiv:2606.13497},
  year={2026}
}