Instructions to use google/gemma-3-27b-it with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use google/gemma-3-27b-it with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="google/gemma-3-27b-it") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForImageTextToText processor = AutoProcessor.from_pretrained("google/gemma-3-27b-it") model = AutoModelForImageTextToText.from_pretrained("google/gemma-3-27b-it") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- HuggingChat
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use google/gemma-3-27b-it with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "google/gemma-3-27b-it" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "google/gemma-3-27b-it", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/google/gemma-3-27b-it
- SGLang
How to use google/gemma-3-27b-it with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "google/gemma-3-27b-it" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "google/gemma-3-27b-it", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "google/gemma-3-27b-it" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "google/gemma-3-27b-it", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use google/gemma-3-27b-it with Docker Model Runner:
docker model run hf.co/google/gemma-3-27b-it
Wrong answer by gemma3-27b-it : what is the difference betweem each: google/gemma-3-27b-it, google/gemma-3-27b-pt
Wrong answer given below
The difference between google/gemma-3-27b-it and google/gemma-3-27b-pt lies in the language they are primarily trained on and optimized for:
google/gemma-3-27b-it: This model is specifically trained and optimized for Italian. It excels at tasks in Italian, including generation, understanding, and translation to and from Italian. While it can handle other languages, its performance will be significantly better in Italian. "it" stands for Italian.
google/gemma-3-27b-pt: This model is specifically trained and optimized for Portuguese. It excels at tasks in Portuguese, including generation, understanding, and translation to and from Portuguese. While it can handle other languages, its performance will be significantly better in Portuguese. "pt" stands for Portuguese.
Hi @Waisullah ,
pt refers to pre-trained Variant means trained on large amounts of general data like web pages, books, code, etc. which has general knowledge but doesn't know how to follow instructions precisely.
it refers to Instruction-Tuned Variant can Fine-tuned on natural langauage instructions which can better at following commands and behaving like a helpful assistant.
For an better understanding, could you please refer to the below example:
prompt = "Summarize this sentence: 'The quick brown fox jumps over the lazy dog.'"
Pre-trained model might respond = “The quick brown fox jumps over the lazy dog.”
Instruction-tuned model respond = “A fox jumps over a dog.”
Instruction-tuned model knows you are expecting a summary, not a repeat.
Thank you.
The difference between google/gemma-3-27b-it and google/gemma-3-27b-pt lies in the language they are primarily trained on and optimized for:
google/gemma-3-27b-it: This model is specifically trained and optimized for Italian. It excels at tasks in Italian, including generation, understanding, and translation to and from Italian. While it can handle other languages, its performance will be significantly better in Italian. "it" stands for Italian.
google/gemma-3-27b-pt: This model is specifically trained and optimized for Portuguese. It excels at tasks in Portuguese, including generation, understanding, and translation to and from Portuguese. While it can handle other languages, its performance will be significantly better in Portuguese. "pt" stands for Portuguese.
You are joking, right?
Hi @Waisullah ,
pt refers to pre-trained Variant means trained on large amounts of general data like web pages, books, code, etc. which has general knowledge but doesn't know how to follow instructions precisely.
it refers to Instruction-Tuned Variant can Fine-tuned on natural langauage instructions which can better at following commands and behaving like a helpful assistant.
For an better understanding, could you please refer to the below example:
prompt = "Summarize this sentence: 'The quick brown fox jumps over the lazy dog.'" Pre-trained model might respond = “The quick brown fox jumps over the lazy dog.” Instruction-tuned model respond = “A fox jumps over a dog.”Instruction-tuned model knows you are expecting a summary, not a repeat.
Thank you.
Thanks yeah I understand I was just pointing out the wrong output generated by the model.