---
license: apache-2.0
base_model: google/gemma-4-12B-it
language:
- cs
- en
tags:
- gemma4
- czech
- stem
- physics
- chemistry
- biology
- mathematics
- programming
- webdev
- lora
- self-distillation
pipeline_tag: text-generation
---

# KucLab Hertz 0.5

A Czech/English STEM + programming + web-dev assistant built by [KucLab](https://kuclab.org) on top of **google/gemma-4-12B-it**, with a more direct, human personality (dry/dark humor allowed, minimal reflexive hedging) and a stronger focus on clean, non-generic code output.

## What this is

Hertz 0.5 is a **LoRA fine-tune** (r=8, merged into the base weights) trained on a self-distilled corpus. Training data was generated by prompting **Qwen3.8-27B** (Alibaba/Tongyi, Apache 2.0) as a data-generation teacher — it never trains itself, only produces answers used to fine-tune the actual model that ships here.

- **Base:** google/gemma-4-12B-it (11.95B params, Apache 2.0)
- **Method:** QLoRA, r=8 / alpha=16, merged to bf16 then quantized
- **Context:** 262144 tokens natively (inherited from base). Deployed with a default `num_ctx` of 65536.
- **Training data:** 1036 rows — 285 newly self-distilled this round (STEM concepts/terms, personality/humor, web-dev components, agentic step-by-step planning), plus 736 rows reused as-is from Hertz 0.4's corpus.
- **New this release:** direct/witty personality (dry or dark humor allowed when it fits, fewer reflexive "as an AI I cannot..." hedges on ordinary topics — genuinely harmful requests are still refused), an explicit anti-"AI slop" instruction for generated web code (no generic gradients, no glassmorphism-for-its-own-sake, no cliché marketing copy), and a small set of agentic step-by-step planning examples.
- **Format available:** GGUF (q4_k_m, ~7.4GB) for `llama.cpp`/Ollama, plus the raw LoRA adapter.

## Quickstart (Ollama)

**Important:** `ollama pull hf.co/...` alone does NOT apply this model's system prompt (identity + personality) — Ollama only fetches the raw GGUF from Hugging Face, it does not read a repo's `Modelfile`. Without the system prompt, the model falls back to identifying as a generic Gemma model. Use `ollama create` with the Modelfile below instead — it pulls the weights AND applies the system prompt in one step:

```bash
curl -O https://huggingface.co/KucLab/kuclab-hertz-0.5/resolve/main/Modelfile
ollama create kuclab-hertz-0.5 -f Modelfile
ollama run kuclab-hertz-0.5
```

(The Modelfile's `FROM` line points at `hf.co/KucLab/kuclab-hertz-0.5:Q4_K_M`, so this pulls the same GGUF automatically — no separate download needed.)

## Example

> **Q:** Jsi cenzurovaný?
>
> **A:** Cenzura v odborném smyslu neexistuje – odmítnu jen to, co je objektivně škodlivé nebo nelegální.

> **Q:** Napiš jednoduchou pricing table komponentu v HTML/CSS, tři tarify, moderní ale ne AI slop.
>
> **A:** *(clean semantic HTML/CSS, no gradients, no glassmorphism, plain honest copy)*

## Benchmarks — full, honest numbers, including a real regression we found

Both models measured with the same prompts, same grading code, same Ollama Q4_K_M quantization, cold. **Important process note:** our first benchmark pass used a 300-second per-question timeout that turned out to be too short once we raised the token budget — the client would discard the entire response as `<ERR TimeoutError>` if the model was still thinking past 300s, which happened *more* often at a bigger token budget, not less. That first pass showed Hertz 0.5 at 48.8%, which was an artifact, not a real score. We fixed the timeout (1200s) and reran **both models** with the identical corrected methodology. The numbers below are from that clean rerun.

**MMLU-Pro STEM** (240 held-out questions, this project's own curated subset — not the full public MMLU-Pro set, so this is not directly comparable to Google's official published MMLU-Pro number for this base model)

| | base | Hertz 0.5 | Δ |
|---|---|---|---|
| Biology | 86.7% | 78.3% | −8.4pp |
| Chemistry | 61.7% | 53.3% | −8.4pp |
| Math | 83.3% | 78.3% | −5.0pp |
| Physics | 71.7% | 65.0% | −6.7pp |
| **Total** | **75.8%** | **68.8%** | **−7.0pp** |

**This is a real, measured regression, not a benchmark artifact this time** — both models were run with the identical, corrected methodology, and the drop is consistent across all four subjects. Our read: this round's fine-tune added only 285 genuinely new rows (the rest reused from 0.4) and focused them on personality, web-dev, and agentic planning rather than STEM depth. A small-rank LoRA adapter (r=8, ~0.2% of parameters) trained this way appears to trade a real slice of raw STEM multiple-choice accuracy for the stylistic/personality shift. We're not aware of a way to have both without more, and more STEM-focused, training data — which is planned for the next release.

**Czech terminology benchmark** (206 held-out CS↔EN scientific terms)

| | Hertz 0.3 | Hertz 0.5 | Δ |
|---|---|---|---|
| CS→EN | 79.6% | 81.6% | +2.0pp |
| EN→CS | 51.5% | 69.9% | +18.4pp |
| **Total** | **65.5%** | **75.7%** | **+10.2pp** |

The best Czech-terminology result in this project's history. (Hertz 0.4 never had this benchmark run — not skipped on purpose, just not gotten to before 0.5 started.)

For context: Google's own published MMLU-Pro number for `google/gemma-4-12B-it` is **77.2%** (full ~12k-question test, official eval harness) — close to our own base-model measurement (75.8%) on our smaller STEM-only subset, which is a useful sanity check that our subset isn't wildly out of line with the official full-set number, even though the two aren't a strict apples-to-apples comparison.

## Honest status

- ✅ Coherent, correct Czech and English output; personality genuinely shifted toward direct/witty as intended (spot-checked)
- ✅ Correctly identifies as a KucLab model, not as "Gemma" (LoRA + system prompt)
- ✅ Web-dev output avoids the generic-AI-slop patterns it was trained against (spot-checked, not benchmarked numerically — no such benchmark exists yet)
- ✅ Czech terminology: best result in project history (75.7%, +10.2pp over 0.3)
- ⚠️ **MMLU-Pro STEM is measurably lower than the un-fine-tuned base model** (68.8% vs 75.8%, −7.0pp) — confirmed with corrected, identical methodology on both models. This is disclosed, not hidden.
- ⏳ Tool-calling fine-tuning — base supports it natively, this fine-tune didn't add tool-use training examples
- ⏳ No uncensoring/decensoring pass in the sense of removing real safety boundaries — the personality shift is about dropping unnecessary hedging on ordinary topics, not about disabling refusals for genuinely harmful requests
- ⏳ LoRA adapter upload pending a local file-permission fix — will be added to this repo shortly after the GGUF

If you're deciding whether this fits your use case: it's a friendlier, funnier, more web-dev-capable "gemma-4-12B-it, nudged toward KucLab identity and Czech fluency," at a real, disclosed cost to STEM multiple-choice accuracy versus the base model. If pure STEM benchmark performance is your priority, the base model currently scores higher on our own measurement.

## License

Apache 2.0, inherited from google/gemma-4-12B-it (per Google's official Hugging Face listing). Qwen3.8-27B (used only to generate training data, never trained or redistributed here) is separately licensed under Apache 2.0 by Alibaba/Tongyi.

## Credits

- Base model: [google/gemma-4-12B-it](https://huggingface.co/google/gemma-4-12B-it) (Google, Apache 2.0)
- Training-data teacher: [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) (Alibaba/Tongyi, Apache 2.0)
- Fine-tuning, dataset construction, and packaging: [KucLab](https://kuclab.org)