Instructions to use google/gemma-2-9b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use google/gemma-2-9b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="google/gemma-2-9b")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-9b") model = AutoModelForCausalLM.from_pretrained("google/gemma-2-9b") - Inference
- Local Apps Settings
- vLLM
How to use google/gemma-2-9b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "google/gemma-2-9b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "google/gemma-2-9b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/google/gemma-2-9b
- SGLang
How to use google/gemma-2-9b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "google/gemma-2-9b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "google/gemma-2-9b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "google/gemma-2-9b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "google/gemma-2-9b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use google/gemma-2-9b with Docker Model Runner:
docker model run hf.co/google/gemma-2-9b
Does Gemma 2 9B Support All Listed Languages on the Gemini 1.5 Page?
I noticed on the Google page that Gemini supports the following languages, but the page only mentions Gemini 1.5.
I'm curious to know if Gemma 2 9B are also supports all these languages ?
https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models?hl=en#languages-gemini
Gemini models support the following languages:
Arabic (ar), Bengali (bn), Bulgarian (bg), Chinese simplified and traditional (zh), Croatian (hr), Czech (cs), Danish (da), Dutch (nl), English (en), Estonian (et), Finnish (fi), French (fr), German (de), Greek (el), Hebrew (iw), Hindi (hi), Hungarian (hu), Indonesian (id), Italian (it), Japanese (ja), Korean (ko), Latvian (lv), Lithuanian (lt), Norwegian (no), Polish (pl), Portuguese (pt), Romanian (ro), Russian (ru), Serbian (sr), Slovak (sk), Slovenian (sl), Spanish (es), Swahili (sw), Swedish (sv), Thai (th), Turkish (tr), Ukrainian (uk), Vietnamese (vi).
Hi @i18n-site , The Gemma 2 9B model is trained on a vast dataset of text and code, enabling support for a diverse range of languages.
To determine the language support of the Gemma 2 9B model, you can analyze its vocabulary by searching for specific language alphabets or tokens. For example, to verify if the model supports Telugu, you can check if a particular Telugu character is present in the vocabulary. If it is found, you can conclude that the model supports the Telugu language. Similarly, you can perform this check for other languages by searching for their respective characters or tokens in the model's vocabulary. Kindly find the below screenshot. Thank you.
Gemma 2 Only supports English based on the following link:
https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models?hl=en#gemma-models
