Instructions to use hohai/gemma4-my-model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use hohai/gemma4-my-model with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("hohai/gemma4-my-model") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Unsloth Studio
How to use hohai/gemma4-my-model with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for hohai/gemma4-my-model to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for hohai/gemma4-my-model to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for hohai/gemma4-my-model to start chatting
Load model with FastModel
pip install unsloth from unsloth import FastModel model, tokenizer = FastModel.from_pretrained( model_name="hohai/gemma4-my-model", max_seq_length=2048, ) - MLX LM
How to use hohai/gemma4-my-model with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "hohai/gemma4-my-model"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "hohai/gemma4-my-model" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hohai/gemma4-my-model", "messages": [ {"role": "user", "content": "Hello"} ] }'
File size: 1,316 Bytes
48f5957 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | {
"audio_seq_length": 750,
"image_processor": {
"do_convert_rgb": true,
"do_normalize": false,
"do_rescale": true,
"do_resize": true,
"image_mean": [
0.0,
0.0,
0.0
],
"image_processor_type": "Gemma4ImageProcessor",
"image_seq_length": 280,
"image_std": [
1.0,
1.0,
1.0
],
"max_soft_tokens": 280,
"patch_size": 16,
"pooling_kernel_size": 3,
"resample": 3,
"rescale_factor": 0.00392156862745098,
"size": {
"height": 224,
"width": 224
}
},
"image_seq_length": 280,
"processor_class": "Gemma4Processor",
"video_processor": {
"default_fps": 2.0,
"do_normalize": true,
"do_rescale": true,
"image_mean": [
0.0,
0.0,
0.0
],
"image_std": [
1.0,
1.0,
1.0
],
"max_soft_tokens": 70,
"num_frames": 32,
"patch_size": 16,
"pooling_kernel_size": 3,
"rescale_factor": 0.00392156862745098,
"video_processor_type": "Gemma4VideoProcessor"
},
"feature_extractor": {
"feature_extractor_type": "Gemma4AudioFeatureExtractor",
"sampling_rate": 16000,
"num_mel_filters": 128,
"fft_length": 512,
"hop_length": 160,
"chunk_duration": 8.0,
"overlap_duration": 1.0
},
"audio_ms_per_token": 40
} |