README.md

---
license: gemma
base_model: google/gemma-4-26B-A4B-it
datasets:
  - bigcode/self-oss-instruct-sc2-exec-filter-50k
library_name: transformers
pipeline_tag: text-generation
tags:
  - code
  - coding-assistant
  - qlora
  - unsloth
model-index:
  - name: gemma-coder
    results:
      - task:
          type: text-generation
          name: Code generation
        dataset:
          type: code-eval
          name: remote-agent-dev-platform coding_eval
        metrics:
          - type: pass@1
            value: 1.0000
            name: pass@1 (Python/JS/React/Go/Java/Swift)
---

# gemma-coder

Coding-focused fine-tune of [`google/gemma-4-26B-A4B-it`](https://huggingface.co/google/gemma-4-26B-A4B-it)
(**Gemma 4 26B A4B**, an MoE with ~4B active params), produced automatically by the
weekly retrain pipeline in [remote-agent-dev-platform](https://github.com/Monibee-Fudgekins/remote-agent-dev-platform).
**Last updated: 2026-08-05 15:14 UTC** · run mode: `full` · promoted: **True**.

## Model description
QLoRA fine-tune of google/gemma-4-26B-A4B-it specialized for coding assistance. It is the
default agent model for the remote-agent-dev-platform (served via vLLM on Modal).

## Intended uses & limitations
- **Intended:** code generation and assistance in Python, JavaScript/React, Go, Java,
  and Swift, inside a sandboxed agent that runs/tests the output.
- **Not intended:** safety-critical use, or running generated code unreviewed.
- **Limitations:** a small, free-tier-trained model — it can produce incorrect or
  insecure code. Always review and test. Quality tracks the training data, which is
  still being built out.

## Training data
- Dataset: [`bigcode/self-oss-instruct-sc2-exec-filter-50k`](https://huggingface.co/datasets/bigcode/self-oss-instruct-sc2-exec-filter-50k)

## Training procedure
- Method: QLoRA (Unsloth), 4-bit base, LoRA r=8 / alpha=16 on
  attention only, lr 2e-05,
  max seq len 768, optimizer adamw_8bit.
- Progress: **cycle 1 — 2831 / 4000 steps** (trained
  in weekly ~8h chunks on Kaggle's free 2×T4, resuming each week; training is
  continuous — a finished cycle rolls into the next).

## Evaluation
Sandboxed multi-language **pass@1** harness (`finetune/evaluate.py`): the model
completes functions that are then compiled/run against unit tests. Languages whose
toolchain is unavailable are skipped.

**Overall pass@1: 100.00%** over 16 executed problems
(4 skipped). Promotion threshold: 80%.

| language | passed / run | pass@1 |
|---|---|---|
| go | 4/4 | 100.00% |
| java | 4/4 | 100.00% |
| javascript | 4/4 | 100.00% |
| python | 4/4 | 100.00% |
| swift | 0/0 | skipped (no toolchain) |

## How to use
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Monibee-Fudgekins/gemma-coder")
model = AutoModelForCausalLM.from_pretrained("Monibee-Fudgekins/gemma-coder", device_map="auto")
msgs = [{"role": "user", "content": "Write a Python function that reverses a string."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(ids, max_new_tokens=256)[0]))
```


## Provenance
Generated by `finetune/kaggle/run.py` in [https://github.com/Monibee-Fudgekins/remote-agent-dev-platform](https://github.com/Monibee-Fudgekins/remote-agent-dev-platform); see that repo for
the full training + eval pipeline.