Instructions to use ImAXISOG/iris-gemma-4-e2b-it-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use ImAXISOG/iris-gemma-4-e2b-it-gguf with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M # Run inference directly in the terminal: llama cli -hf ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M # Run inference directly in the terminal: llama cli -hf ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M
Use Docker
docker model run hf.co/ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use ImAXISOG/iris-gemma-4-e2b-it-gguf with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ImAXISOG/iris-gemma-4-e2b-it-gguf" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ImAXISOG/iris-gemma-4-e2b-it-gguf", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M
- Ollama
How to use ImAXISOG/iris-gemma-4-e2b-it-gguf with Ollama:
ollama run hf.co/ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M
- Unsloth Desktop
- Pi
How to use ImAXISOG/iris-gemma-4-e2b-it-gguf with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use ImAXISOG/iris-gemma-4-e2b-it-gguf with Docker Model Runner:
docker model run hf.co/ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M
- Lemonade
How to use ImAXISOG/iris-gemma-4-e2b-it-gguf with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M
Run and chat with the model
lemonade run user.iris-gemma-4-e2b-it-gguf-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use ImAXISOG/iris-gemma-4-e2b-it-gguf with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use ImAXISOG/iris-gemma-4-e2b-it-gguf with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Iris Gemma 4 E2B (GGUF)
The writing model inside Iris, an on-device grammar and tone app: "Lighter" in the app. It does two jobs and is trained for nothing else:
- Fix corrects spelling, grammar, capitals and punctuation, and changes nothing else. Your words, short forms, slang and casual voice stay as you typed them. A message with no mistakes comes back unchanged.
- Tones rewrite a message as Professional, Friendly, Shorter, Clearer or Funny, keeping every point, who does what, and the questions as questions.
It reads English, Hindi (Devanagari) and Hinglish (Hindi typed in English letters), and always answers in English.
| Pick this one for | nearly as good in English, weaker in Hindi, smaller and faster. Meant for Macs and PCs with 8 GB of memory (not yet tested on an 8 GB machine). |
|---|---|
| The other one | Best quality (E4B): iris-gemma-4-e4b-it-gguf |
| All Iris models | Iris writing models |
| Files | GGUF, Q4_K_M, with the per-layer word table (per_layer_token_embd) at Q4_K, 2.81 GB (this repo). For Macs: iris-gemma-4-e2b-it-mlx (MLX, 3.72 GB). |
| Based on | Google's google/gemma-4-E2B-it, fine-tuned with LoRA |
| License | Apache 2.0, a modified version of Gemma 4 (see License) |
Which one should I use?
| Your computer | Use | Memory it needs |
|---|---|---|
| Mac with 16 GB or more | Best quality: iris-gemma-4-e4b-it-mlx | 3.6 GB at peak |
| Mac with 8 GB | Lighter: iris-gemma-4-e2b-it-mlx | 2.9 GB at peak |
| Windows PC with 16 GB or more | Best quality: iris-gemma-4-e4b-it-gguf | 3.9 GB, plus the 4.6 GB file kept in memory: 8.4 GB in all |
| Windows PC with 8 GB | Lighter: iris-gemma-4-e2b-it-gguf | 1.9 GB, plus the 2.8 GB file kept in memory: 4.8 GB in all |
If Hindi or Hinglish matters most, use Best quality whenever the computer has 16 GB: on Hindi Fix it is accepted 95% of the time against Lighter's 71%. On Windows the model file counts too: when memory runs short, Windows drops the file from memory and reads it from disk again, and answers slow down. That is why Best quality needs a 16 GB PC even though its own memory is under 4 GB.
The Mac rows are the rule Iris uses at setup. The Windows numbers were measured with llama.cpp on a Mac's processor cores. Neither has been confirmed yet on a real Windows PC or on an 8 GB computer of either kind.
Results
| Gemma 4 E4B-it, untrained | Iris Lighter (E2B) | Iris Best quality (E4B) | |
|---|---|---|---|
| English Fix: accepted (50 drafts) | 52% | 86% | 90% |
| Professional and Friendly: would send (100 answers) | 86% | 88% | 93% |
| Shorter and Clearer: would send (100 answers) | 75% | 86% | 90% |
| Funny: would send (50 answers) | 4% | 56% | 66% |
| Hindi and Hinglish, Fix (into English): accepted (82 drafts) | 91% | 71% | 95% |
| Hindi and Hinglish, tones (into English): would send (240 answers) | 71% | 47% | 76% |
Judged blind by an AI model, answer by answer, on drafts written only for testing (see How it was tested). Gaps under about 8 points are within noise.
The Lighter model against Google's untrained E2B, from earlier judging rounds (so not on the chart): English Fix 86% against 60% (measured on the previous training round, which shares its English lessons), Hindi tones would send 47% against 36%, Hindi Fix 62 to 65% against 63% (no change). For Hindi and Hinglish, use the E4B model.
Examples
Real answers from this file, unedited, to drafts written for this card (temperature 0, Iris's prompts, llama.cpp on a processor).
All 30 answers are in examples.jsonl.
| Draft | hey are we still on for tmrw? i didnt got the slides yet, can u send them before 10 |
| Fix | hey are we still on for tmrw? I didn't get the slides yet, can u send them before 10? |
| Professional | Hi, are we still on for tomorrow? I have not received the slides yet. Could you send them before 10? |
| Friendly | Hey, are we still on for tomorrow? I haven't got the slides yet, so could you send them before 10? Thanks! |
| Shorter | Hey, still on for tmrw? I haven't got the slides yet, can u send them before 10? |
| Clearer | Hey, are we still on for tomorrow? Can you send the slides before 10? I haven't got them yet. |
| Funny | Hey are we still on for tmrw? I didn't get the slides yet, can u send them before 10? |
Hinglish and Hindi come back in English, keeping who does what:
| Draft | Mode | Answer |
|---|---|---|
| bhai kal meeting 11 baje hai, report tum bhej dena main slides bana dunga | Fix | bro the meeting is at 11 tomorrow, you send the report I'll make the slides |
| Professional | The meeting is at 11 tomorrow. Please send the report. I will prepare the slides. | |
| เคฎเคพเคซเคผ เคเฅเคเคฟเค, เคเค เคฎเฅเค เคฅเฅเคกเคผเคพ เคฆเฅเคฐ เคธเฅ เคเคเคเคเคพ, เคเฅเคฐเฅเคจ เคฒเฅเค เคนเฅเฅค | Fix | Sorry, I'll be a little late today, the train is late. |
A complaint stays a complaint, and Funny stays out of it:
| Draft | Mode | Answer |
|---|---|---|
| the new pump arrived damaged, this is the second time this month and nobody from your team has called me back | Professional | The new pump arrived damaged. This is the second time this month, and nobody from your team has called me back. |
| Funny | The new pump arrived damaged, this is the second time this month and nobody from your team has called me back. |
Memory and speed
- Mac (MLX, Apple silicon): peak 2.9 GB (3.3 GB at the longest message Iris takes, 6,000 characters), measured through Iris's own loader after a real download; about 0.2 s for a typical answer on an M5 Pro.
- Windows (GGUF, llama.cpp, processor only): 1.9 GB of its own memory plus the 2.81 GB file, which the system keeps in memory when it has room (4.8 GB in all); about 0.5 s an answer. Suggested: 8 GB of RAM (not yet tested on an 8 GB machine). Measured on a Mac's processor cores with no graphics chip, not yet on a Windows laptop.
How to use
llama.cpp
Tested with llama.cpp 0.9.0 (Homebrew build 12110):
llama-server -hf ImAXISOG/iris-gemma-4-e2b-it-gguf:Q4_K_M -np 1 --jinja --port 8080
curl -s localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
"messages": [
{"role": "system", "content": "<the Fix prompt below>"},
{"role": "user", "content": "<message>\nhey are we still on for tmrw? i didnt got the slides yet, can u send them before 10\n</message>"}],
"temperature": 0, "chat_template_kwargs": {"enable_thinking": false}}'
Turn thinking off (enable_thinking: false), use temperature 0 for Fix, and run one request at a time (-np 1).
Other llama.cpp-based apps (LM Studio, Ollama, Jan) were not tested.
The prompts it was trained with
The model learned Iris's short instructions, with thinking turned off. Other prompts work but were not measured.
Fix (English)
The draft goes in the user turn as <message>\n...\n</message>.
Proofread the message between <message> and </message>. Correct only clear mistakes in spelling, grammar, capitalization and punctuation. Change nothing else: keep the writer's words, style, short forms, emoji, numbers and line breaks. Copy placeholders like {{1}} exactly once. If there are no mistakes, return the message exactly as it is. Reply with only the corrected message.
Fix for a message written in Hindi or Hinglish
Iris sends this system prompt, then one example pair (user: maine invoice bhej diya hai, kal tak payment aa jayega,
assistant: I've sent the invoice, the payment will come by tomorrow), then the draft as it is.
The user's message is written in Hindi, usually in Latin letters (Hinglish), sometimes in Devanagari. Translate it into natural English.
- Keep the meaning, facts, names, numbers, and requests exactly. Keep commitments as firm as written.
- Keep who does what: "bhej dena" asks the reader to send; "bhej dunga" means the writer will send.
- Keep the tone: a casual message stays casual. Don't add greetings or sign-offs.
- Placeholders like {{1}} stand for names, links, and code. Copy every placeholder exactly once, in the same place.
- Reply with only the English translation: no Hindi words, no quotes, no labels, no explanation.
Tones
The draft goes in the user turn as it is. {tone} is the tone's name in lower case.
Rewrite the message in a {tone} tone. {instruction}
Keep every point of the draft and add nothing. Keep who does what, and keep questions as questions. Fix grammar and spelling. Copy placeholders like {{1}} exactly once. No em dashes. Reply with only the rewritten message.
- Professional: Sound clear and professional, like a capable colleague. Not stiff or flowery: no "I hope this finds you well", no "kindly". Professional changes how a thing is said, never whether it is said: a complaint, a disagreement, a refusal, frustration, an apology, thanks, and a courtesy like "I know it's late" all stay, put calmly and plainly. Never drop a point because it is blunt, and never soften a complaint into a request. Keep the point, not the wording: say each point in plain professional words, turning slang, sarcasm, insults and filler words like "really" or "super" into calm, direct language, so it reads as a colleague wrote it, not as the draft with its spelling fixed.
- Friendly: Sound warm and easygoing, like a teammate people enjoy working with. Use contractions and plain, everyday words. Show warmth with words, never emoji: every rewrite gets one warm touch, such as a thanks, a word of acknowledgment, or a friendly opener like "Hey" or "Quick heads-up:". When the message is routine, one light, good-natured turn of phrase can make it friendlier; leave it out for bad news, apologies, complaints, or anything urgent, and never use sarcasm or joke about anyone. At most one exclamation mark. No filler openers like "Hope you're doing well". Be warm in the wording, never by hedging: if it says something will happen, it still will.
- Shorter: Make it as short as possible by cutting filler words and repetition. Every fact, question, request, and condition stays.
- Clearer: Restructure it so the reader knows what to do on the first read: put the main point or request in the first sentence, then the details. Split long sentences, use everyday words, and use a short list when there are several items or steps.
- Funny: Add one light, good-natured touch of humor beside the message and change nothing else: keep the message's own words, and invent no facts or details. Never joke about people, mistakes, or anything sensitive. When humor doesn't fit (bad news, an apology, a complaint), only tidy the message.
How it was made
| Starting point | Google's google/gemma-4-E2B-it, as the 4-bit MLX copy mlx-community/gemma-4-E2B-it-qat-4bit (what Iris used before), with thinking switched off in the chat template |
|---|---|
| Lessons | 8,851 distinct examples (10,741 rows: some lessons appear more than once), 408 held back to check progress |
| Where lessons came from | Drafts and answers written and reviewed with AI models; Hindi and Hinglish translations reviewed by AI reviewers who corrected tense, idiom and who-does-what. None is taken from the test drafts, and lessons that resembled a test draft were dropped. Not released. |
| Method | LoRA on every layer: rank 16, scale 20, dropout 0.05; loss on the answers only (prompts masked) |
| Schedule | One pass, 2,686 steps, batch 4, learning rate 5e-5 with cosine decay to 2e-6, 20 warm-up steps, up to 2,048 tokens |
| Hardware | One Apple M5 Pro Mac, 1 h 6 min, with mlx-lm |
| Held-back loss | 0.268 |
| Afterwards | Adapter merged into the weights; 4-bit (group size 64), with the MLP layers at 8-bit as in the quantization-aware copy it was trained from; audio weights removed (picture weights kept so stock loaders work) |
| GGUF | Converted with llama.cpp's convert_hf_to_gguf.py, quantized to Q4_K_M, with the per-layer word table (per_layer_token_embd) at Q4_K |
How it was tested
- Drafts: messages people type at work and to friends and family, written only for testing and never shown to whoever wrote the lessons: 50 English drafts (Fix and all five tones), 82 Hindi and Hinglish drafts for Fix, and 240 tone answers on Hindi and Hinglish drafts.
- Answers: through the same code the app uses. The trained models at temperature 0 with Iris's short prompts; the untrained model as Iris ran it before, with Iris's longer prompts (worked examples included) and its usual settings.
- Judge: an AI model, blind: the answers of all three models to a draft were shown together in shuffled order, with no model names, and judged against written rules.
- Fix "accepted": no real mistake left and nothing changed beyond trivial tidying (a reworded phrase, an expanded short form or a lost casual voice counts against it). For Hindi drafts: an accurate, natural English version.
- Tones "would send": the writer could send it in place of the draft: the same meaning, the tone asked for, and it reads naturally.
- The GGUF file was judged on the answers that differ from the MLX model's. Against the MLX model, counted over two judging rounds (MLX against the standard Q4_K_M, then that against this smaller file): English tones 2 fewer good answers of 200, Hindi and Hinglish Fix 1 fewer of 82, English Fix 1 more of 50 (the 4 answers that differ in the first round were read by hand and came out even). Funny and Hindi tones were not compared with the MLX model; against the standard Q4_K_M this file gave 2 fewer good Funny answers of 50 and the same on Hindi tones.
Limitations
- Answers in English only. Hindi and Hinglish drafts are read and answered in English.
- Not a chat assistant. It was trained for Fix and the five tones; questions or other tasks in a draft are rewritten, not answered. Picture input was not trained or tested.
- Funny is the weakest tone: good enough to send 56% of the time in the test. Hindi is weaker on this model than on the E4B model.
- Judged by an AI model, not by people. Its rules are strict, but it can still be wrong.
- Not tested yet: an 8 GB Mac or a real Windows laptop, long documents (memory only), custom tones, a writer's own style rules, and the drafter in llama.cpp.
- Iris checks every answer before showing it (for example, that names, links and code kept as placeholders come back exactly once). Used on its own, the model has no such checks.
About Iris
Iris is a grammar and tone app that runs entirely on your computer: press a shortcut in any app to fix or rewrite what you wrote. macOS now, Windows planned.
License
Apache License 2.0 (LICENSE). This is a modified version of Google's Gemma 4 E2B-it
(google/gemma-4-E2B-it, Gemma 4 license):
fine-tuned with LoRA on writing lessons, merged, quantized, audio weights removed.
- Downloads last month
- 157
4-bit
Model tree for ImAXISOG/iris-gemma-4-e2b-it-gguf
Collection including ImAXISOG/iris-gemma-4-e2b-it-gguf
Evaluation results
- accepted (%) on Iris blind test, English Fix (50 drafts)self-reported86.000
- would send (%) on Iris blind test, Professional and Friendly (100 answers)self-reported88.000
- would send (%) on Iris blind test, Shorter and Clearer (100 answers)self-reported86.000
- would send (%) on Iris blind test, Funny (50 answers)self-reported56.000
- accepted (%) on Iris blind test, Hindi and Hinglish, Fix (into English) (82 drafts)self-reported71.000
- would send (%) on Iris blind test, Hindi and Hinglish, tones (into English) (240 answers)self-reported47.000



