Image-Text-to-Text
Transformers
Safetensors
inkling2
conversational
Mixture of Experts
Eval Results
8-bit precision
modelopt
Instructions to use thinkingmachines/Inkling-2-Small with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use thinkingmachines/Inkling-2-Small with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="thinkingmachines/Inkling-2-Small") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("thinkingmachines/Inkling-2-Small") model = AutoModelForMultimodalLM.from_pretrained("thinkingmachines/Inkling-2-Small", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- HuggingChat
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use thinkingmachines/Inkling-2-Small with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "thinkingmachines/Inkling-2-Small" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling-2-Small", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/thinkingmachines/Inkling-2-Small
- SGLang
How to use thinkingmachines/Inkling-2-Small with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "thinkingmachines/Inkling-2-Small" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling-2-Small", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "thinkingmachines/Inkling-2-Small" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling-2-Small", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use thinkingmachines/Inkling-2-Small with Docker Model Runner:
docker model run hf.co/thinkingmachines/Inkling-2-Small
Commit History
6 days ago
-
Add SWE-Bench Pro (full) and SWE-bench Multimodal results
Verified signatureBoth from the longer evaluation table at https://thinkingmachines.ai/model-card/inkling-2-small/ (not in the README). Note: the SWE-Bench Pro v2 (Full) score (97.7) is higher than the Hard-subset score (86.3); please double-check it before merging.48f5e83 -
Add evaluation results
Verified signature# Add evaluation results for thinkingmachines/Inkling-2-Small ## Summary This PR adds evaluation results from the model card's benchmark table for `thinkingmachines/Inkling-2-Small` to the `.eval_results/` directory, following the [Hugging Face Hub evaluation-results specification](https://huggingface.co/docs/hub/eval-results). ## Benchmarks Added - [DeepSWE v1.1](https://huggingface.co/datasets/datacurve/deep-swe?eval_result=thinkingmachines/Inkling-2-Small&leaderboard_task_id=deep_swe): 72.1 - [Terminal-Bench-Science 0.1](https://huggingface.co/datasets/harborframework/terminal-bench-science?eval_result=thinkingmachines/Inkling-2-Small&leaderboard_task_id=terminalbenchscience_0_1): 5.7 - [SWE-Bench Pro v2 (Hard)](https://huggingface.co/datasets/ScaleAI/SWE-bench_Pro?eval_result=thinkingmachines/Inkling-2-Small&leaderboard_task_id=SWE_Bench_Pro_hard): 86.3 - [ScreenSpot-Pro](https://huggingface.co/datasets/likaixin/ScreenSpot-Pro?eval_result=thinkingmachines/Inkling-2-Small&leader…a6251a1 -
Keep nested tool-call argument order in the chat template
Verified signatureTool-call arguments are sorted by top-level name only, and each argument's value keeps its own key order, as the reference renderer writes them. Keys inside object-valued arguments were previously sorted too.aac9741
7 days ago
-
Update README.md
Verified signaturee242a97 -
Adding transformers as the library name (#1)
- Adding transformers as the library name (9b5b4ee19f176833cd153e238182fc4c87c5f7ae) Co-authored-by: Aritra Roy Gosthipaty <ariG23498@users.noreply.huggingface.co>b0e7afb -
upload model
78cb6c0