Image-Text-to-Text
Transformers
Safetensors
inkling2
conversational
Mixture of Experts
Eval Results
8-bit precision
modelopt
Instructions to use thinkingmachines/Inkling-2-Small with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use thinkingmachines/Inkling-2-Small with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="thinkingmachines/Inkling-2-Small") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("thinkingmachines/Inkling-2-Small") model = AutoModelForMultimodalLM.from_pretrained("thinkingmachines/Inkling-2-Small", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- HuggingChat
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use thinkingmachines/Inkling-2-Small with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "thinkingmachines/Inkling-2-Small" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling-2-Small", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/thinkingmachines/Inkling-2-Small
- SGLang
How to use thinkingmachines/Inkling-2-Small with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "thinkingmachines/Inkling-2-Small" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling-2-Small", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "thinkingmachines/Inkling-2-Small" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling-2-Small", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use thinkingmachines/Inkling-2-Small with Docker Model Runner:
docker model run hf.co/thinkingmachines/Inkling-2-Small
Commit History
4 days ago
-
[marstin-dev] drop sage1.1 for mistral4l on validation set; update tab headline metric
Verified signatureUpdates the evaluation table to match https://thinkingmachines.ai/model-card/inkling-2-small/: - Tax Agent Bench uses its new headline metric (e.g. Inkling 2 Small 38.9%, DeepSeek-V4.1-Flash-1017 44.4%); best and second-best highlights are unchanged. - SAGE v1.1: Mistral Large 4's score is removed. No other changes to the README.5f67157
6 days ago
-
Update README.md
Verified signature8361e7a -
Add community evaluation results for Terminal-Bench 2.1, Terminal-Bench, SWE-bench Multilingual, GPQA Diamond, HMMT February 2026 (#5)
- Add community evaluation results for Terminal-Bench 2.1, Terminal-Bench, SWE-bench Multilingual, HMMT February 2026 (e41e091266f2bccb69f35bd91ccb182336f899c0) - Add GPQA Diamond evaluation result (1aed104292493eb8a313ef76e94d12c9171020c0) Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>ef5bf64 -
Add evaluation results (#4)
- Add evaluation results (a6251a1144f44605c2c5771a0a97e96965922cec) - Add SWE-Bench Pro (full) and SWE-bench Multimodal results (48f5e831c53d9f0a5522b2988e21e8c956ce05ba) Co-authored-by: Nathan Habib <SaylorTwift@users.noreply.huggingface.co>77b7815 -
Add minimal quantization config to correct Hub parameter count (#3)
- Add minimal quantization config to correct Hub parameter count (04062d4c310be465b29b3352bce2b8a390aabfed) Co-authored-by: ben burtenshaw <burtenshaw@users.noreply.huggingface.co>3cd8c5a -
Keep nested tool-call argument order in the chat template
Verified signatureTool-call arguments are sorted by top-level name only, and each argument's value keeps its own key order, as the reference renderer writes them. Keys inside object-valued arguments were previously sorted too.aac9741
7 days ago
-
Update README.md
Verified signaturee242a97 -
Adding transformers as the library name (#1)
- Adding transformers as the library name (9b5b4ee19f176833cd153e238182fc4c87c5f7ae) Co-authored-by: Aritra Roy Gosthipaty <ariG23498@users.noreply.huggingface.co>b0e7afb -
upload model
78cb6c0