Instructions to use InternRobotics/VLAC-Cut with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use InternRobotics/VLAC-Cut with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="InternRobotics/VLAC-Cut")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("InternRobotics/VLAC-Cut")
model = AutoModelForMultimodalLM.from_pretrained("InternRobotics/VLAC-Cut", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use InternRobotics/VLAC-Cut with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "InternRobotics/VLAC-Cut"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InternRobotics/VLAC-Cut",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/InternRobotics/VLAC-Cut

SGLang

How to use InternRobotics/VLAC-Cut with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "InternRobotics/VLAC-Cut" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InternRobotics/VLAC-Cut",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "InternRobotics/VLAC-Cut" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InternRobotics/VLAC-Cut",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use InternRobotics/VLAC-Cut with Docker Model Runner:
```
docker model run hf.co/InternRobotics/VLAC-Cut
```

VLAC-Cut / examples /reference_outputs /example_03.jsonl

futurefantasy

Update reference outputs to prediction-only preview videos

9dabf65 verified about 1 month ago

Raw

History Blame

2.09 kB

{"example_id": "example_03", "bucket": "test_expert_seen", "global_episode_id": "ARX-data/human-data-0114/20260112-yuanshuaijun/20260111-204118-put-triangular-beaker-onto-tripod-good2/videos/chunk-000/observation.images.front/episode_000002", "task_instruction": "将三角烧杯放在三脚架上。", "task_description": "将三角烧杯放在三脚架上。\n爪夹准备移动：0%\n爪夹开始移动：10%\n爪夹靠近三角烧杯：20%\n爪夹抓取三角烧杯：40%\n爪夹合拢并靠近三脚架：60%\n爪夹移动到三脚架的正上方：80%\n爪夹松开，三角烧杯落在三脚架上：100%", "prompt_variant": "chunk_all", "input_sample_hz": 2.0, "input_frame_indices_2hz": [0, 15, 30, 45, 60, 75, 90, 105, 120, 135, 150, 165], "input_timestamps_sec_2hz": [0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5], "input_frame_count_2hz": 12, "decoded_video_stats": {"decoded_frame_count": 172.0, "decoded_avg_fps": 30.0}, "prompt": "任务描述和具体规划: 将三角烧杯放在三脚架上。\n爪夹准备移动：0%\n爪夹开始移动：10%\n爪夹靠近三角烧杯：20%\n爪夹抓取三角烧杯：40%\n爪夹合拢并靠近三脚架：60%\n爪夹移动到三脚架的正上方：80%\n爪夹松开，三角烧杯落在三脚架上：100%\n\n请根据任务描述和具体规划，找到并逐点生成视频中的关键动作点和相应的进度标注。输出格式要求：每个关键点一行，格式为：\n时间: X.Xs, 进度: Y%\n\n请严格按照上述格式输出，不要输出额外说明。", "response": "时间: 0.1s, 进度: 0%\n时间: 0.6s, 进度: 10%\n时间: 1.1s, 进度: 20%\n时间: 1.6s, 进度: 30%\n时间: 2.3s, 进度: 40%\n时间: 3.4s, 进度: 60%\n时间: 4.1s, 进度: 80%\n时间: 5.3s, 进度: 100%", "pred_curve_parse_ok": true, "pred_curve_point_times_sec": [0.1, 0.6, 1.1, 1.6, 2.3, 3.4, 4.1, 5.3], "pred_curve_point_progress": [0.0, 10.0, 20.0, 30.0, 40.0, 60.0, 80.0, 100.0], "video_path": "examples/example_03/episode.mp4", "preview_video_path": "examples/reference_outputs/example_03_pred_progress.mp4"}