Instructions to use hohai/gemma4-my-model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use hohai/gemma4-my-model with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("hohai/gemma4-my-model") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Unsloth Studio
How to use hohai/gemma4-my-model with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for hohai/gemma4-my-model to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for hohai/gemma4-my-model to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for hohai/gemma4-my-model to start chatting
Load model with FastModel
pip install unsloth from unsloth import FastModel model, tokenizer = FastModel.from_pretrained( model_name="hohai/gemma4-my-model", max_seq_length=2048, ) - MLX LM
How to use hohai/gemma4-my-model with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "hohai/gemma4-my-model"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "hohai/gemma4-my-model" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hohai/gemma4-my-model", "messages": [ {"role": "user", "content": "Hello"} ] }'
| { | |
| "audio_seq_length": 750, | |
| "image_processor": { | |
| "do_convert_rgb": true, | |
| "do_normalize": false, | |
| "do_rescale": true, | |
| "do_resize": true, | |
| "image_mean": [ | |
| 0.0, | |
| 0.0, | |
| 0.0 | |
| ], | |
| "image_processor_type": "Gemma4ImageProcessor", | |
| "image_seq_length": 280, | |
| "image_std": [ | |
| 1.0, | |
| 1.0, | |
| 1.0 | |
| ], | |
| "max_soft_tokens": 280, | |
| "patch_size": 16, | |
| "pooling_kernel_size": 3, | |
| "resample": 3, | |
| "rescale_factor": 0.00392156862745098, | |
| "size": { | |
| "height": 224, | |
| "width": 224 | |
| } | |
| }, | |
| "image_seq_length": 280, | |
| "processor_class": "Gemma4Processor", | |
| "video_processor": { | |
| "default_fps": 2.0, | |
| "do_normalize": true, | |
| "do_rescale": true, | |
| "image_mean": [ | |
| 0.0, | |
| 0.0, | |
| 0.0 | |
| ], | |
| "image_std": [ | |
| 1.0, | |
| 1.0, | |
| 1.0 | |
| ], | |
| "max_soft_tokens": 70, | |
| "num_frames": 32, | |
| "patch_size": 16, | |
| "pooling_kernel_size": 3, | |
| "rescale_factor": 0.00392156862745098, | |
| "video_processor_type": "Gemma4VideoProcessor" | |
| }, | |
| "feature_extractor": { | |
| "feature_extractor_type": "Gemma4AudioFeatureExtractor", | |
| "sampling_rate": 16000, | |
| "num_mel_filters": 128, | |
| "fft_length": 512, | |
| "hop_length": 160, | |
| "chunk_duration": 8.0, | |
| "overlap_duration": 1.0 | |
| }, | |
| "audio_ms_per_token": 40 | |
| } |