Image-Text-to-Text
Transformers
Safetensors
qwen3_vl_moe
robotics
embodied-ai
video-understanding
progress-estimation
reward-modeling
qwen3-vl
conversational
Instructions to use InternRobotics/VLAC-Cut with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use InternRobotics/VLAC-Cut with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="InternRobotics/VLAC-Cut") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("InternRobotics/VLAC-Cut") model = AutoModelForMultimodalLM.from_pretrained("InternRobotics/VLAC-Cut", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use InternRobotics/VLAC-Cut with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "InternRobotics/VLAC-Cut" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "InternRobotics/VLAC-Cut", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/InternRobotics/VLAC-Cut
- SGLang
How to use InternRobotics/VLAC-Cut with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "InternRobotics/VLAC-Cut" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "InternRobotics/VLAC-Cut", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "InternRobotics/VLAC-Cut" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "InternRobotics/VLAC-Cut", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use InternRobotics/VLAC-Cut with Docker Model Runner:
docker model run hf.co/InternRobotics/VLAC-Cut
Add examples/example_01/metadata.json
Browse files
examples/example_01/metadata.json
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"example_id": "example_01",
|
| 3 |
+
"bucket": "test_nonexpert_seen",
|
| 4 |
+
"global_episode_id": "ARX-data/human-data-1128/20251126-150505-good-cocket-cylindrical-container/videos/chunk-000/observation.images.front/episode_000007",
|
| 5 |
+
"main_path": "ARX-data/human-data-1128/20251126-150505-good-cocket-cylindrical-container/videos/chunk-000/observation.images.front/episode_000007",
|
| 6 |
+
"task_instruction": "抓取插座放入电饭锅的内锅中",
|
| 7 |
+
"task_description": "抓取插座放入电饭锅的内锅中:\n开始移动:0%\n爪夹靠近插座:20%\n爪夹抓取插座:40%\n爪夹夹紧插座靠近内锅:60%\n爪夹夹紧插座落下放到内锅:80%\n爪夹松开,插座落入内锅中:100%",
|
| 8 |
+
"fps": 30.0,
|
| 9 |
+
"is_expert": false,
|
| 10 |
+
"num_frames": 338,
|
| 11 |
+
"final_gt_progress": 100.0,
|
| 12 |
+
"video_path": "examples/example_01/episode.mp4",
|
| 13 |
+
"benchmark_progress_type": "dense_kinematic_progress",
|
| 14 |
+
"benchmark_dense_progress": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -0.2, -0.4, -0.5, -0.7, -0.9, -1.1, -1.2, -1.4, -1.6, -1.8, -2.0, -2.1, -2.3, -2.5, -2.7, -2.9, -3.0, -3.2, -3.4, -3.6, -3.8, -3.9, -4.1, -4.3, -4.5, -4.6, -4.8, -5.0, -5.1, -5.2, -5.3, -5.5, -5.6, -5.7, -5.8, -5.9, -6.0, -6.2, -6.3, -6.4, -6.5, -6.6, -6.7, -6.8, -7.0, -7.1, -7.2, -7.3, -7.4, -7.5, -7.7, -7.8, -7.9, -8.0, -7.2, -6.4, -5.7, -4.9, -4.1, -3.3, -2.5, -1.7, -1.0, -0.2, 0.6, 1.4, 2.2, 3.0, 3.7, 4.5, 5.3, 6.1, 6.9, 7.7, 8.4, 9.2, 10.0, 10.3, 10.5, 10.8, 11.1, 11.3, 11.6, 11.8, 12.1, 12.4, 12.6, 12.9, 13.2, 13.4, 13.7, 13.9, 14.2, 14.5, 14.7, 15.0, 15.2, 15.3, 15.5, 15.7, 15.8, 16.0, 16.2, 16.3, 16.5, 16.7, 16.8, 17.0, 17.2, 17.3, 17.5, 17.7, 17.8, 18.0, 18.2, 18.3, 18.5, 18.7, 18.8, 19.0, 19.2, 19.3, 19.5, 19.7, 19.8, 20.0, 20.6, 21.3, 21.9, 22.6, 23.2, 23.9, 24.5, 25.2, 25.8, 26.5, 27.1, 27.7, 28.4, 29.0, 29.7, 30.3, 31.0, 31.6, 32.3, 32.9, 33.5, 34.2, 34.8, 35.5, 36.1, 36.8, 37.4, 38.1, 38.7, 39.4, 40.0, 40.3, 40.7, 41.0, 41.4, 41.7, 42.1, 42.4, 42.8, 43.1, 43.4, 43.8, 44.1, 44.5, 44.8, 45.2, 45.5, 45.9, 46.2, 46.6, 46.9, 47.2, 47.6, 47.9, 48.3, 48.6, 49.0, 49.3, 49.7, 50.0, 50.3, 50.6, 50.9, 51.1, 51.4, 51.7, 52.0, 52.3, 52.6, 52.9, 53.1, 53.4, 53.7, 54.0, 54.3, 54.6, 54.9, 55.1, 55.4, 55.7, 56.0, 56.3, 56.6, 56.9, 57.1, 57.4, 57.7, 58.0, 58.3, 58.6, 58.9, 59.1, 59.4, 59.7, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 80.9, 81.7, 82.6, 83.5, 84.3, 85.2, 86.1, 87.0, 87.8, 88.7, 89.6, 90.4, 91.3, 92.2, 93.0, 93.9, 94.8, 95.7, 96.5, 97.4, 98.3, 99.1, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0],
|
| 15 |
+
"benchmark_semantic_anchors": [{"frame": 42, "progress": 0.0}, {"frame": 70, "progress": -5.0}, {"frame": 96, "progress": -8.0}, {"frame": 119, "progress": 10.0}, {"frame": 138, "progress": 15.0}, {"frame": 168, "progress": 20.0}, {"frame": 199, "progress": 40.0}, {"frame": 228, "progress": 50.0}, {"frame": 263, "progress": 60.0}, {"frame": 283, "progress": 80.0}, {"frame": 306, "progress": 100.0}],
|
| 16 |
+
"benchmark_progress_source": {
|
| 17 |
+
"bucket": "test_nonexpert_seen",
|
| 18 |
+
"global_episode_id": "ARX-data/human-data-1128/20251126-150505-good-cocket-cylindrical-container/videos/chunk-000/observation.images.front/episode_000007"
|
| 19 |
+
}
|
| 20 |
+
}
|