Instructions to use jgebbeken/gemma-4-coder-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jgebbeken/gemma-4-coder-gguf with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="jgebbeken/gemma-4-coder-gguf",
	filename="gemma-4-E4b-it.BF16-mmproj.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use jgebbeken/gemma-4-coder-gguf with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf jgebbeken/gemma-4-coder-gguf:BF16
# Run inference directly in the terminal:
llama-cli -hf jgebbeken/gemma-4-coder-gguf:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf jgebbeken/gemma-4-coder-gguf:BF16
# Run inference directly in the terminal:
llama-cli -hf jgebbeken/gemma-4-coder-gguf:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf jgebbeken/gemma-4-coder-gguf:BF16
# Run inference directly in the terminal:
./llama-cli -hf jgebbeken/gemma-4-coder-gguf:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf jgebbeken/gemma-4-coder-gguf:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf jgebbeken/gemma-4-coder-gguf:BF16

Use Docker

docker model run hf.co/jgebbeken/gemma-4-coder-gguf:BF16

LM Studio
Jan
Ollama
How to use jgebbeken/gemma-4-coder-gguf with Ollama:
```
ollama run hf.co/jgebbeken/gemma-4-coder-gguf:BF16
```

Unsloth Studio new

How to use jgebbeken/gemma-4-coder-gguf with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for jgebbeken/gemma-4-coder-gguf to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for jgebbeken/gemma-4-coder-gguf to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for jgebbeken/gemma-4-coder-gguf to start chatting

Pi new

How to use jgebbeken/gemma-4-coder-gguf with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf jgebbeken/gemma-4-coder-gguf:BF16

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "jgebbeken/gemma-4-coder-gguf:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use jgebbeken/gemma-4-coder-gguf with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf jgebbeken/gemma-4-coder-gguf:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default jgebbeken/gemma-4-coder-gguf:BF16

Run Hermes

hermes

Docker Model Runner
How to use jgebbeken/gemma-4-coder-gguf with Docker Model Runner:
```
docker model run hf.co/jgebbeken/gemma-4-coder-gguf:BF16
```

Lemonade

How to use jgebbeken/gemma-4-coder-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull jgebbeken/gemma-4-coder-gguf:BF16

Run and chat with the model

lemonade run user.gemma-4-coder-gguf-BF16

List all available models

lemonade list

New Model coming soon

by jgebbeken - opened Apr 19

Discussion

jgebbeken

Owner Apr 19

9,000+ downloads on my Gemma 4 Coder model. Honestly didn't expect this at all — thank you all so much! 🎉 It's really pushed me to keep going. Next month I'll be dropping a Gemma-4-26B-A4B version using the same MagicCoder-style training. Same approach, just a much larger model. Hope you guys are looking forward to it! 🚀
If you want to help fund the next training run, I have a Ko-fi ☕ https://ko-fi.com/D1D61Y2FGJ

lastprophet

Apr 19

Hi, I wanted to send you an email, but I can't seem to find your contact information anywhere. I’ve been testing Gemma on its own, but I haven't been able to edit anything beyond basic code. I tested your model, and it works very well. I wanted to ask if, besides Gemma, there is any other more robust model for agents. If you could share your email address, I’d be happy to share more details on how I configured it. Thanks, it was very helpful.

jgebbeken

Owner Apr 19

Hi, I wanted to send you an email, but I can't seem to find your contact information anywhere. I’ve been testing Gemma on its own, but I haven't been able to edit anything beyond basic code. I tested your model, and it works very well. I wanted to ask if, besides Gemma, there is any other more robust model for agents. If you could share your email address, I’d be happy to share more details on how I configured it. Thanks, it was very helpful.

Hey, thanks so much for the kind words — really glad it's working well for you! For agents I'd honestly check out MiniMax M2.7 first, specifically for complex agent workflows and works really well and extremely cheap. You can also pair it with Hermes Agent by Nous Research which has has native M2.7 support. For Local models can go Qwen3-Coder and Devstral you want something fully local. Feel free to drop more details here in the comments, happy to help out! For the email, I am afraid that isn't something I want to give out at this time. Thank you for your understanding.

lastprophet

Apr 20

Hi Josh, thanks for the recommendations!

I wanted to share that, so far, only the version you uploaded and the official one are working correctly for me with OpenClaw. I’ve been testing several other models, but none of them seem to yield the same results.

I’m currently testing Qwen3.5-9b and managed to get it connected. However, in quick tests, it sometimes doesn't follow instructions perfectly, and it runs a bit slower than Gemma. I’ll keep testing it to see if I can optimize it.

Regarding your other suggestions, they are a bit too heavy for me. Due to my current setup, I can only run models that are around 6 GB or smaller.

Let's keep the conversation here. Thanks again for the insight!

jgebbeken

Owner Apr 20

I think from my understanding thst qwen3.5 done a lot of overthinking where Qwen3.6 at least didn't do it as much. I am glad that mine is working for you. Enjoy.

lastprophet

Apr 21

I just tried this version of qwen/qwen2.5-coder-7b-instruct-gguf and it worked perfectly for me too. Thanks a lot for the recommendation! I'll be looking forward to your updates.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment