Instructions to use google/gemma-4-31B-it with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use google/gemma-4-31B-it with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="google/gemma-4-31B-it")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("google/gemma-4-31B-it")
model = AutoModelForMultimodalLM.from_pretrained("google/gemma-4-31B-it")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
AMD Developer Cloud
Local Apps Settings

vLLM

How to use google/gemma-4-31B-it with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "google/gemma-4-31B-it"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "google/gemma-4-31B-it",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/google/gemma-4-31B-it

SGLang

How to use google/gemma-4-31B-it with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "google/gemma-4-31B-it" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "google/gemma-4-31B-it",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "google/gemma-4-31B-it" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "google/gemma-4-31B-it",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use google/gemma-4-31B-it with Docker Model Runner:
```
docker model run hf.co/google/gemma-4-31B-it
```

Reviews of Gemma 4

#92

by Juanoto2012 - opened Apr 29

Discussion

Juanoto2012

Apr 29

What do you think about Gemma 4?

Juanoto2012

Apr 29

In my personal opinion, Gemma 4 is one of the most optimized models. With just 16GB of RAM it's incredibly fast, and even with 8GB it's still decent—I tested it on my phone. Good job, Google!

joggyback

Jun 4

I hope 124b-it can be released. That will be a great news

Juanoto2012

24 days ago

yes

kenleo

23 days ago

I have been using Gemma-4-31B-it and Qwen3.6-27B for a while. I also used Qwen3.6-35B-A3B. In my opinion, Gemma-4-31B-it is like an intelligent and honest person who gets things done quietly, while Qwen3.6-27B is like a workhorse who, though not-so-smart and a little dull, is willing to put in ten times the effort to achieve results.

I believe Alibaba is committed to competing with Google, and open-sourcing a moderately capable model is undoubtedly driven by two reasons: 1. If the model isn't open-sourced to benefit peripheral users, limited computing power would have to serve more people, leading to resource constraints and impacting the experience of high-paying users. Allowing ordinary people to use a model that covers basic needs without paying helps shift costs to the user side; 2. Open-sourcing a moderately capable model can significantly suppress competitors, causing cost-sensitive users to abandon competitors' model subscriptions, while simultaneously enhancing Alibaba's social influence. Overall, this approach is beneficial for both large model companies and users. However, as technology iterates over time, large model companies not only need to improve the capabilities of cutting-edge models but also, strategically, should promptly update and supplement open-source models to ensure users remain loyal to their open-source ecosystem, cultivate user habits, and provide differentiated capabilities on cutting-edge models, thereby driving users from private deployment to a hybrid ecosystem of private deployment and subscription coexistence. The current closed-source model landscape is becoming increasingly closed and self-contradictory. Lower-ranking competitors should quickly adopt a two-pronged approach: leveraging this opportunity to advance closed-source model development while simultaneously open-sourcing more advanced models to benefit both paying and casual users, maintaining a good user experience, and garnering more user support.

I feel the current capability gap between mid-range open-source models available to ordinary users and cutting-edge closed-source models is too large, hindering user retention. Over time, users will objectively evaluate comparable open-source models and switch to truly excellent ones, rather than relying on company-released performance comparison charts that don't reflect the end-user experience. Truly powerful model companies must be at the forefront in both the open-source and closed-source arenas.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment