---
license: apache-2.0
base_model: Qwen/Qwen2.5-14B-Instruct
language:
- cs
- en
tags:
- qwen2
- czech
- stem
- physics
- chemistry
- biology
- mathematics
- lora
- self-distillation
pipeline_tag: text-generation
---

# KucLab Hertz 0.3

A Czech/English STEM assistant built by [KucLab](https://kuclab.org) on top of **Qwen2.5-14B-Instruct**, fine-tuned to be sharper on physics, chemistry, biology and mathematics in Czech — while carrying forward the base model's general capability, 128k-token context reach, and native tool-calling support.

## What this is

Hertz 0.3 is a **LoRA fine-tune** (r=8, merged into the base weights) trained on a self-distilled corpus of Czech/English STEM concepts, worked problems, and formatting examples. Training data was generated by prompting **Qwen3.8-27B** (Alibaba/Tongyi, Apache 2.0) — a stronger reasoning model used purely as a data-generation teacher, never trained itself — and used to fine-tune the actual 14B model that ships here.

- **Base:** Qwen/Qwen2.5-14B-Instruct (14.7B params, Apache 2.0)
- **Method:** QLoRA, r=8 / alpha=16, merged to fp16 then quantized
- **Context:** extended to 128k via YaRN rope scaling (native 32k → 131072, factor 4.0)
- **Training data:** 242 self-distilled rows — Czech/English STEM concepts & terminology, worked problems with step-by-step reasoning, Markdown/code formatting examples, identity
- **Format available:** GGUF (q4_k_m, ~9GB) for `llama.cpp`/Ollama, plus the raw LoRA adapter for anyone who wants to merge it onto their own copy of the base model

## Why a fine-tune instead of a bigger model

Qwen2.5-14B fits comfortably on a single consumer/cloud GPU (23GB) for both inference and QLoRA training. Rather than chase parameter count, this project focuses the model's existing capacity on a specific gap: strong, natural Czech STEM terminology and step-by-step worked solutions, which the base model handles less fluently out of the box.

## Quickstart (Ollama)

**Important:** `ollama pull hf.co/...` alone does NOT apply this model's system prompt (identity + personality) — Ollama only fetches the raw GGUF from Hugging Face, it does not read a repo's `Modelfile`. Without the system prompt, the model falls back to identifying as a generic Gemma model. Use `ollama create` with the Modelfile below instead — it pulls the weights AND applies the system prompt in one step:

```bash
curl -O https://huggingface.co/KucLab/kuclab-hertz-0.3/resolve/main/Modelfile
ollama create kuclab-hertz-0.3 -f Modelfile
ollama run kuclab-hertz-0.3
```

(The Modelfile's `FROM` line points at `hf.co/KucLab/kuclab-hertz-0.3:Q4_K_M`, so this pulls the same GGUF automatically — no separate download needed.)

Or build from the Modelfile in this repo with the bundled GGUF.

## Example

> **Q:** Kolik je 17 × 23? Ukaž postup.
>
> **A:** 17 × 23 = 17 × (20 + 3) = 340 + 51 = **391**

> **Q:** Vysvětli druhý Newtonův zákon.
>
> **A:** *(explains F = dp/dt, in fluent Czech, with the formula and its meaning)*

## Benchmarks

Measured against the same base model (`qwen2.5:14b-instruct`, Q4_K_M via Ollama), same prompts, same grading code, both models cold.

**MMLU-Pro STEM** (240 held-out questions, this project's own curated subset — not a claim of parity with the standard published MMLU-Pro leaderboard number)

| | base | Hertz 0.3 | Δ |
|---|---|---|---|
| Biology | 78.3% | 75.0% | −3.3pp |
| Chemistry | 71.7% | 71.7% | 0pp |
| Math | 88.3% | 86.7% | −1.6pp |
| Physics | 71.7% | 80.0% | **+8.3pp** |
| **Total** | **77.5%** | **78.3%** | **+0.8pp** |

**Czech scientific terminology** (206 held-out terms, both translation directions, this project's own curated set)

| | base | Hertz 0.3 | Δ |
|---|---|---|---|
| CS → EN | 79.6% | 79.6% | 0pp |
| EN → CS | 47.6% | 51.5% | **+3.9pp** |
| **Total** | **63.6%** | **65.5%** | **+1.9pp** |

Read this as: no regression on general STEM reasoning, a real (if modest) gain on Czech terminology — which is exactly what the fine-tune targeted. Physics moved the most; biology and math moved slightly the other way. These are two custom benchmarks built for this project, not standardized public leaderboards — useful for before/after comparison on this exact model, not for cross-model bragging rights.

## Honest status

This is a small-scale, single-GPU fine-tuning project, not a frontier lab release. What's verified:

- ✅ Coherent, correct Czech and English output (spot-checked: arithmetic, physics, chemistry, terminology)
- ✅ Correctly identifies as a KucLab model, not as "Qwen" (LoRA + system prompt)
- ✅ 128k context window configured and loadable (long-context *quality* not yet independently verified with held-out long documents)
- ✅ MMLU-Pro STEM and Czech terminology benchmarked against base (see above) — no regression, modest gains

What's **not** done yet:

- ⏳ Tool-calling fine-tuning — the base model supports function calling natively, but this fine-tune did not add tool-use training examples
- ⏳ No uncensoring/decensoring pass — safety behavior is inherited from the base model as-is
- ⏳ No standardized public benchmarks (GSM8K, HumanEval, official MMLU, etc.) — only the two project-specific evals above

If you're deciding whether this fits your use case: treat it as "Qwen2.5-14B-Instruct, nudged toward Czech STEM fluency and KucLab identity," not as a categorically different or more powerful model than its base.

## License

Apache 2.0, inherited from Qwen2.5-14B-Instruct. This repository includes a LoRA adapter and a merged/quantized derivative of that base model; see `LICENSE` for full terms. Qwen3.8-27B (used only to generate training data, never trained or redistributed here) is separately licensed under Apache 2.0 by Alibaba/Tongyi.

## Credits

- Base model: [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct) (Alibaba Cloud, Apache 2.0)
- Training-data teacher: [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) (Alibaba/Tongyi, Apache 2.0)
- Fine-tuning, dataset construction, and packaging: [KucLab](https://kuclab.org)