Instructions to use inference-optimization/gemma-4-unified-1.5B-small with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use inference-optimization/gemma-4-unified-1.5B-small with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="inference-optimization/gemma-4-unified-1.5B-small")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("inference-optimization/gemma-4-unified-1.5B-small")
model = AutoModelForMultimodalLM.from_pretrained("inference-optimization/gemma-4-unified-1.5B-small", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use inference-optimization/gemma-4-unified-1.5B-small with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "inference-optimization/gemma-4-unified-1.5B-small"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inference-optimization/gemma-4-unified-1.5B-small",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/inference-optimization/gemma-4-unified-1.5B-small

SGLang

How to use inference-optimization/gemma-4-unified-1.5B-small with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "inference-optimization/gemma-4-unified-1.5B-small" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inference-optimization/gemma-4-unified-1.5B-small",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "inference-optimization/gemma-4-unified-1.5B-small" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inference-optimization/gemma-4-unified-1.5B-small",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use inference-optimization/gemma-4-unified-1.5B-small with Docker Model Runner:
```
docker model run hf.co/inference-optimization/gemma-4-unified-1.5B-small
```

gemma-4-unified-1.5B-small

This is a small version of google/gemma-4-12B-it created for testing and development. It keeps all hyperparameters of the base model identical and only reduces the number of layers, so it loads in vLLM with the same characteristics as the base.

Model Details

Base Model: google/gemma-4-12B-it
Architecture: gemma4_unified / Gemma4UnifiedForConditionalGeneration (omni: text + vision + audio)
Total Parameters: 1.53B (1,525,009,920)
Activated Parameters: 1.53B (dense model, all parameters active per token)

Configuration Changes

The following parameters were reduced from the original model:

Parameter	Base	Small
`text_config.num_hidden_layers`	48	2
`text_config.layer_types`	5x sliding_attention : 1x full_attention, 8 periods	1x sliding_attention + 1x full_attention
`text_config.per_layer_config`	full-attn override (head_dim 512, num_key_value_heads 1) at layers 05,11,...,47	same override at layer 1 (the full-attention layer)

All other hyperparameters are identical to the base: hidden_size 3840, intermediate_size 15360, 16 attention heads / 8 KV heads (head_dim 256) on sliding layers, head_dim 512 / 1 KV head on full-attention layers, attention_k_eq_v (no v_proj on full-attention layers), sliding_window 1024, vocab 262144, tied embeddings, gelu_pytorch_tanh MLP, per-type rope parameters (proportional partial-rotary full attention, default sliding), final_logit_softcapping 30, use_bidirectional_attention "vision".

The minimal covering set of 2 layers (one per distinct attention type, base ordering sliding-before-full preserved) was used because the fixed dimensions force a larger model: the tied 262144x3840 embedding alone is ~1.01B, and each text layer costs ~224M (sliding) / ~242M (full) parameters. A full 5:1 period (6 layers) would be ~2.4B. The vision and audio towers of this architecture are encoder-free projection embedders with no layers, so there were no tower layer counts to reduce (their hidden sizes and all other hyperparameters are untouched).

Checkpoint Structure

Single model.safetensors file with 40 tensors, using the original base checkpoint naming (model.language_model.*, model.vision_embedder.*, model.embed_vision.embedding_projection.weight, model.embed_audio.embedding_projection.weight), including the per-layer layer_scalar buffers. lm_head is absent because word embeddings are tied, as in the base. Verified against the base checkpoint header fetched over HTTP: 0 missing and 0 extra tensor-name patterns; the layer-0 key set is identical to a base sliding-attention layer and the layer-1 key set is identical to a base full-attention layer (base layer 5), with identical shapes.

Usage

from transformers import Gemma4UnifiedForConditionalGeneration, AutoProcessor

model = Gemma4UnifiedForConditionalGeneration.from_pretrained("inference-optimization/gemma-4-unified-1.5B-small", device_map="auto")
processor = AutoProcessor.from_pretrained("inference-optimization/gemma-4-unified-1.5B-small")

input_ids = processor.tokenizer("According to all known laws", return_tensors="pt").input_ids.to(model.device)
output = model.generate(input_ids, max_new_tokens=20)
print(processor.tokenizer.decode(output[0]))

Creation Process

This model was created using the llm-compressor create-tiny-model skill.

Loaded the base config and applied only the allowed changes (num_hidden_layers 48 -> 2 with layer_types/per_layer_config trimmed consistently).
Built the model with random init (no base weights downloaded) and fixed non-finite initializations.
Fine-tuned on the skill's toy copypasta dataset (text-only) with bf16 until convergence (train loss ~0.05).
Validated the checkpoint structure against the base and the fine-tuned model's perplexity/generation.

The gemma4_unified weight-conversion registry (6 tensor renamings between the released checkpoint naming and the transformers module tree) was kept on save, exactly as a real load of the base records it, so the saved checkpoint uses the original base tensor naming.

Notes

No MTP: this family has no num_nextn_predict_layers in its config; the checkpoint is the full model.
No quantization_config in the base (bf16), so none was stripped.
Validation: perplexity 1.0016 on the validation text (target <= 10), greedy generation reproduces the fine-tuning text, and an image+text multimodal forward through the processor produces finite logits (280 image soft tokens + 5 text tokens).
Fine-tuned weights are uploaded (not the raw random init).

Validation Output

Perplexity: 1.0015603303909302 (target <= 10.0)
Success

Generating sample text:
According to all known laws of aviation, there is no way a bee should be able to fly.

Multimodal forward OK: logits shape (1, 285, 262144), finite=True