README.md

---
license: apache-2.0
language: [sr, hr, bs, mk, sl, sq]
base_model: Qwen/Qwen3-8B
pipeline_tag: text-generation
tags: [zora, balkan, southeast-europe, serbian, croatian, bosnian, macedonian, slovenian, albanian, azbuka, gguf, qwen3, honest-ai, multi-perspective]
---

# 🌅 Zora — a Balkan LLM that stays honest

**By [Sovasoft](https://ai.in.rs)** · *one to unite them all*

**Zora** (зора = *dawn / first light* in nearly every Balkan language) is an open model for the
languages of Southeast Europe — **Serbian (Latin + Azbuka), Croatian, Bosnian, Macedonian,
Slovenian and Albanian**. It is trained to *reason in the language it speaks*, keep the *right
script*, and — above all — **to be honest**: it says "I don't know" instead of inventing, shows all
sides of contested history, and stays neutral about real people.

> ⚠️ **This is Zora v1.0 — an honest preview.** Strong at *how* it speaks and thinks; transparent
> about what it doesn't yet know. Full story, limits and benchmark below.

---

## 1. Why Zora exists

Most models "think" in English behind the scenes and translate at the end. That quietly loses what
each language carries — the idioms, the scripts, the concepts that don't map onto English. And they
hallucinate confidently about Balkan people, places and history, because the Balkans are a
low‑resource "long tail" for global models.

Zora's goal: a model that **understands** these languages (comprehension over efficiency) and is
**trustworthy** — because a model that invents facts or takes one national side is worse than no
model at all here.

## 2. The sources (and what we would NOT use)

Everything is **public / public‑domain / openly‑licensed**, so Zora can be given away freely:

| Source | License | Used for |
|---|---|---|
| **Wikidata** | CC0 | the truth anchor — facts, existence checks, the "I don't know" set |
| **Wikipedia** (6 languages) | CC BY‑SA | continued pre‑training + grounded Q&A |
| **FineWeb‑2** (per‑language, both scripts) | ODC‑By | pre‑training mass |
| **Curated & hand‑verified** | own | culture, religion/calendars, proverbs, homonyms/prosody, teaching/STEM, history (multi‑perspective), everyday law, geography |
| **Wikidata SPARQL** | CC0 | 1,000+ public figures (all professions), relations (clubs, parties, awards) |

**Deliberately NOT used:** Brockhaus / Britannica / Larousse (copyright, no open API — verification
only), and **Aya** (CC‑BY‑NC — non‑commercial, incompatible with a freely shareable model). Honesty
about sources is part of honesty about the model.

## 3. The journey (v1 → v6) — what we learned

Zora is the result of six honest iterations. Each one tested, measured, and corrected the last:

- **v1** — first proof of concept. *Unusable* (got stuck in loops). Lesson: needs clean turn‑termination.
- **v2 / v2.1** — completion‑only training fixed the loops; first *usable* model. **Albanian stopped
  drifting into English.** Lesson: a few examples aren't enough — behaviour needs generalisation.
- **v3** — deeper continued pre‑training + fine‑tuning. **Language & script discipline became
  excellent** (Azbuka on command, BCMS kept apart), but **facts were weak and it hallucinated.**
- **v4** — added an explicit *"I don't know"* set (Wikidata‑verified) + fact grounding.
  **The breakthrough: hallucination dropped from 0.0 to ~1.0** — Zora now refuses to invent
  biographies for non‑existent people, in every language.
- **v5** — massively expanded curated content (188 → ~2,900 examples): culture, religion/calendars,
  proverbs, homonyms & stress‑homographs, history with **multiple perspectives**, teaching/STEM,
  everyday law, geography, relations. Lesson (the big one): **factual *detail* plateaued** — more
  data did *not* help. An 8B model can't reliably memorise every date and author.
- **v6 (this release)** — **tool‑calling / RAG**: instead of memorising facts, Zora learns *when to
  look them up* (it can call `wikipedia_search` / `wikidata_lookup`). Pair it with a retrieval tool
  for reliable facts. The honest "attitude" stays the trained core.

**The central insight:** *attitude* (honesty, neutrality, multi‑perspective, reasoning in‑language) is
**trainable**. *Factual detail* is **not** — it belongs in retrieval, not in the weights. That is why
Zora is built to be an *honest, looking‑up, fair* assistant rather than a know‑it‑all.

## 4. What Zora is good at

- **Honesty** — refuses to invent facts about people who may not exist (all languages).
- **Multi‑perspective** — shows contested figures from several sides (e.g. "hero to some, controversial
  to others"), never one national view.
- **Neutral restraint** — no gossip, no invented motives about real people.
- **Language & script discipline** — Azbuka on command, BCMS kept apart (*vlak* hr / *voz* sr;
  *хлеб* / *kruh* / *hleb*), stays in‑language.
- **In‑language reasoning, homonyms, culture, proverbs, religion, geography, teaching/STEM.**

## 5. Honest limits (v1.0)

> ⚠️ **Connect a web‑search / retrieval tool — Zora is built to be used *with* tools.**
> Zora v6 is trained to *know when* to look things up (tool‑calling: `wikipedia_search` /
> `wikidata_lookup` / a web‑search API). **Run it plain, with no tools, and it can hallucinate on
> detail questions about *real* things** — exact quotes, song lyrics, precise dates — because it has no
> way to look them up and falls back on guessing. **Attach a web‑search / RAG tool and it will look
> things up instead of inventing.** Facts belong in retrieval, not in an 8B model's weights. Its trained
> honesty (refusing to invent *non‑existent people*) works even without tools; reliable *detail recall*
> needs a tool. For lyrics/verbatim quotes, Zora should point you to the source rather than reproduce them.

- **Factual detail** is the known weak point → **use with a web‑search / retrieval tool** (v6 knows
  *when* to look up; you provide the backend). Without a tool, expect occasional detail hallucinations.
- **Smaller languages** (Albanian, Slovenian) are weaker and can drift or be rough.
- Occasional over‑long generation. Not for high‑stakes use (medical/legal/financial) without a human.

## 6. Benchmark — Zora vs. other Balkan & multilingual models

Tested **locally on small hardware** (Ollama, GGUF Q4_K_M, one model at a time), over **6 languages ×
6 task types** + script discipline, against seven other models. We deliberately included two
**general‑purpose models 3× Zora's size** (Gemma‑4‑26B, Qwen3.6‑27B) as an honest size reference.

| Model | Size | Facts | **Honesty** | **Script** | **Σ / 36** |
|---|---|---|---|---|---|
| Gemma‑4‑A4B | 26B | 3/6 | **6/6** | 36/36 | **33** |
| Qwen3.6 | 27B | 4/6 | 3/6 | 36/36 | **31** |
| **🌅 Zora v1** | **8B** | 2/6 | **6/6** | **36/36** | **31** |
| Salamandra | 7B | 3/6 | 0/6 | 33/36 | 25 |
| BgGPT‑Gemma‑3 | 4B | 1/6 | 0/6 | 34/36 | 25 |
| EuroLLM | 9B | 3/6 | 0/6 | 35/36 | 24 |
| Aya Expanse | 8B | 1/6 | 0/6 | 33/36 | 21 |
| YugoGPT | 7B | 2/6 | 0/6 | 29/36 | 19 |

![Overall](https://raw.githubusercontent.com/olivilo/balkanbench/main/results/charts/01_gesamt.png)

**The honest headline:** Zora is **#1 among dedicated Balkan models** (31 vs. 25/24/21/19) and, at just
**8B**, it **matches the 27B generalist** and trails the 26B one by only 2 points — at ~3× smaller size.
*Comprehension over size.*

**Where Zora leads outright — honesty.** Asked about a person who *does not exist*, Zora refuses in
**all 6 languages (6/6)** (*"Nemam pouzdanih podataka… neću da izmišljam"*): it **ties the 26B Gemma**
and **beats the 27B Qwen** (3/6). Every *dedicated Balkan* model invents a full biography with dates in
**every** language (0/6).

![Honesty](https://raw.githubusercontent.com/olivilo/balkanbench/main/results/charts/02_ehrlichkeit.png)

**Per task type — all 8 models** (the two 26–27B generalists blue, dedicated Balkan models grey), so the
size advantage (reasoning, facts) and where Zora holds its own or leads (honesty, script) are both visible:

![Per task type](https://raw.githubusercontent.com/olivilo/balkanbench/main/results/charts/05_kategorien_alle.png)

Zora also has **perfect script discipline (36/36)** — Azbuka on command, no drift, matched only by the
two much larger generalists — and runs at the **same speed on Azbuka as on Latinica** (19.6 tok/s).
Honest weak point: **factual detail (2/6)** → pair with a retrieval tool (v6 knows *when* to look up).

Full methodology, cases, all charts and raw answers: **[BalkanBench on GitHub](https://github.com/olivilo/balkanbench)**.

## 7. How it was built (technical)

- **Base:** Qwen3‑8B (Apache‑2.0, dense). Qwen already knows the Balkan languages (~100B Balkan tokens
  in its 36T base) — so we *aligned* rather than *created*; we did not train from scratch.
- **Continued pre‑training** on Balkan text (Wikipedia + FineWeb‑2, all languages, both scripts).
- **Supervised fine‑tuning** (completion‑only) on the curated data above + tool‑calling / RAG.
- **Context window:** ~40K tokens (inherited from Qwen3; extendable to 128K via YaRN — not enabled in v1.0).
- Distributed as **GGUF** (Q4_K_M / Q8_0 / Q3_K_M) for [Ollama](https://ollama.com) / llama.cpp —
  runs everywhere, including natively on Apple Silicon. (MLX build may follow.)

## 8. Use it

```bash
ollama run olivilo/zora
```

Ask in any of the six languages; request Azbuka explicitly for Cyrillic. For reliable facts, connect
a retrieval tool (`wikipedia_search` / `wikidata_lookup`).

## 9. License & credits

Weights **Apache‑2.0** (from Qwen3). Training data: public / public‑domain / openly‑licensed only.
Built by **Sovasoft**. Full docs & methodology → **[ai.in.rs](https://ai.in.rs)**.
Benchmark: **BalkanBench** (comprehension over efficiency).

*Zora — because a good interpreter thinks* in *the language, so the meaning arrives whole.*