Instructions to use google/gemma-3-1b-it with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use google/gemma-3-1b-it with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="google/gemma-3-1b-it")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("google/gemma-3-1b-it")
model = AutoModelForCausalLM.from_pretrained("google/gemma-3-1b-it")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use google/gemma-3-1b-it with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "google/gemma-3-1b-it"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "google/gemma-3-1b-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/google/gemma-3-1b-it

SGLang

How to use google/gemma-3-1b-it with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "google/gemma-3-1b-it" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "google/gemma-3-1b-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "google/gemma-3-1b-it" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "google/gemma-3-1b-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use google/gemma-3-1b-it with Docker Model Runner:
```
docker model run hf.co/google/gemma-3-1b-it
```

Use the model as a sequence classifier

#10

by A123hmed - opened Mar 13, 2025

Discussion

A123hmed

Mar 13, 2025

Hello, I was wondering if you intend on supporting sequence classification like you do for Gemma 2 using "Gemma2ForSequenceClassification".
Do you intent to release "Gemma3ForSequenceClassification"? Because as it stands now, Gemma 2's sequence classifier won't work with Gemma 3 when I try to finetune a Gemma 3 model for sequence classification. Is there any workaround that I can do to allow it to work?

A123hmed

Mar 13, 2025

I did this temporary workaround, in the "modeling_gemma2.py", I import the Gemma 3 text model class as follows:
from ..gemma3.modeling_gemma3 import Gemma3TextModel

and then in "Gemma2ForSequenceClassification" class in the same file, I change this line:
self.model = Gemma2Model(config)
to
self.model = Gemma3TextModel(config)

Schrieffer

Apr 28, 2025

This comment has been hidden (marked as Off-Topic)

Schrieffer

Apr 28, 2025

I did this temporary workaround, in the "modeling_gemma2.py", I import the Gemma 3 text model class as follows:
from ..gemma3.modeling_gemma3 import Gemma3TextModel

and then in "Gemma2ForSequenceClassification" class in the same file, I change this line:
self.model = Gemma2Model(config)
to
self.model = Gemma3TextModel(config)

It's good.
But is the result of Gemma2ForSequenceClassification normal as a reward model?

lkv

Google org Aug 26, 2025

•

edited Aug 26, 2025

Hi ,

You're right—until recently, there wasn't official support for sequence classification with Gemma 3, and workarounds like the one you mentioned (swapping Gemma2Model with Gemma3TextModel) were necessary.

The model will be mostly same as Gemma2 Sequence Classification, with the only difference that it will accept pixel values as input. That means we need a base model for multimodal Gemma3, which adding here (#37033). After that can add Gemma3ForSequenceClassification.

Kindly follow this reference issue . if you have concerns let us know will assist on this.

Thank you.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment