Instructions to use KRLabsOrg/squeez-2b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use KRLabsOrg/squeez-2b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="KRLabsOrg/squeez-2b")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("KRLabsOrg/squeez-2b")
model = AutoModelForImageTextToText.from_pretrained("KRLabsOrg/squeez-2b")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use KRLabsOrg/squeez-2b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "KRLabsOrg/squeez-2b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KRLabsOrg/squeez-2b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/KRLabsOrg/squeez-2b

SGLang

How to use KRLabsOrg/squeez-2b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "KRLabsOrg/squeez-2b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KRLabsOrg/squeez-2b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "KRLabsOrg/squeez-2b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KRLabsOrg/squeez-2b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use KRLabsOrg/squeez-2b with Docker Model Runner:
```
docker model run hf.co/KRLabsOrg/squeez-2b
```

squeez-2b

Commit History

Add extractive alternative section, restore library_name

9d38282
verified

adaamko commited on 10 days ago

Add library_name to metadata (#1)

f97498e

adaamko

nielsr HF Staff commited on 24 days ago

Add arXiv paper link and citation (2604.04979)

3a4c2e6
verified

adaamko commited on 29 days ago

Add mascot image and thumbnail back to model card

85f4123
verified

adaamko commited on Apr 4

Update model card: 2-decimal metrics, 27 tool types, qualitative examples, match paper

37964d9
verified

adaamko commited on Apr 4

Update README.md

2a03adb
verified

adaamko commited on Mar 17

Add related work context

0a80056
verified

adaamko commited on Mar 16

Add Kimi K2 baseline results

8f5689d
verified

adaamko commited on Mar 16

Simplify results table to P/R/F1/Compression

95a9c82
verified

adaamko commited on Mar 16

Upload README.md with huggingface_hub

e137a82
verified

adaamko commited on Mar 16

Upload folder using huggingface_hub

51b4c8b
verified

adaamko commited on Mar 16

initial commit

a8a8e5e
verified

adaamko commited on Mar 16

Commit History

Add extractive alternative section, restore library_name 9d38282 verified

Add library_name to metadata (#1) f97498e

Add arXiv paper link and citation (2604.04979) 3a4c2e6 verified

Add mascot image and thumbnail back to model card 85f4123 verified

Update model card: 2-decimal metrics, 27 tool types, qualitative examples, match paper 37964d9 verified

Update README.md 2a03adb verified

Add related work context 0a80056 verified

Add Kimi K2 baseline results 8f5689d verified

Simplify results table to P/R/F1/Compression 95a9c82 verified

Upload README.md with huggingface_hub e137a82 verified

Upload folder using huggingface_hub 51b4c8b verified

initial commit a8a8e5e verified

Add extractive alternative section, restore library_name

9d38282
verified

Add library_name to metadata (#1)

f97498e

Add arXiv paper link and citation (2604.04979)

3a4c2e6
verified

Add mascot image and thumbnail back to model card

85f4123
verified

Update model card: 2-decimal metrics, 27 tool types, qualitative examples, match paper

37964d9
verified

Update README.md

2a03adb
verified

Add related work context

0a80056
verified

Add Kimi K2 baseline results

8f5689d
verified

Simplify results table to P/R/F1/Compression

95a9c82
verified

Upload README.md with huggingface_hub

e137a82
verified

Upload folder using huggingface_hub

51b4c8b
verified

initial commit

a8a8e5e
verified