| --- |
| language: |
| - or |
| license: other |
| library_name: transformers |
| tags: |
| - odia |
| - text-generation |
| - kalinga-series |
| - unfinished |
| - mystery |
| - evaluated |
| --- |
| |
| # 🏛️ KALINGA — Experiment No. 01 |
|
|
| *One language. Done properly.* |
|
|
| ## The frontier forgot Odia. |
|
|
| 40+ million speakers. A literary tradition older than most languages the frontier |
| has ever shipped. And the biggest labs in the world trained on 200 languages — |
| and left this one out, as if it had never existed. |
|
|
| Kalinga exists because we refuse to accept that. |
|
|
| This is **Experiment No. 01**. The first of a series that will not stop. |
|
|
| --- |
|
|
| ## What we found, and what we kept |
|
|
| We started with **28,549,762** rows of raw Odia. |
|
|
| More than half of it was the same text wearing different clothes. We removed it |
| all. What remains is signal — and nothing else: |
|
|
| | | | |
| |---|---| |
| | **Rows kept** | **12,438,948** — nothing repeated, nothing padded | |
| | **Total text** | **~5.9 billion characters** of Odia | |
| | **English-aligned** | **48.7 %** — real alignments, not flags | |
| | **Per-row** | median 203 · mean 474 characters | |
| | **Quality gate** | ≥ 20 chars · ≥ 10 % Odia script · 63 shards · 3.7 GB | |
| | **Composition** | monolingual 68.9 % · pretrain 15.4 % · parallel 7.7 % · instruction 5.3 % · QA 1.5 % · web · encyclopedic · speech · sentiment | |
|
|
| *The frontier's approach:* 200 languages, two minutes each. |
|
|
| *Kalinga's approach:* **one language, 5.9 billion characters, nothing repeated.** |
|
|
| The corpus has never been published anywhere. While licensing of legacy sources |
| is resolved, it stays private — this model is the closest thing to it that exists |
| in public. |
|
|
| --- |
|
|
| ## The receipts |
|
|
| The first eval numbers exist. They are not flattering. That is the point. |
|
|
| Evaluated on [**Odia Eval Benchmark**](https://huggingface.co/datasets/MaelisResearch/odia-eval-benchmark) |
| (60 rows per task × 7 tasks = 420 rows, greedy decoding, max 96 new tokens, 2× T4): |
|
|
| | Task | Kalinga-01 | sarvam-1 (base) | |
| |---|---|---| |
| | multiple_choice (acc) | 0.0 % | 13.3 % | |
| | QA span F1 | 0.0 | 0.03 | |
| | math (acc) | 0.0 % | 0.0 % | |
| | classification (acc) | 0.0 % | 0.0 % | |
| | NER token F1 | 0.0 | 0.14 | |
| | translation chrF | 0.096 | 0.04 | |
| | generation | logged | logged | |
| |
| **What the numbers say:** Kalinga-01 emits `<reserved_token>` ids in **every |
| one of the 420 generations** — a sign the checkpoint's output layer and |
| embeddings are misaligned with its tokenizer, i.e. the experiment is |
| *unfinished* (as tagged), not merely undertrained. The base model it was built |
| on, sarvam-1, at least produces coherent Odia. Kalinga does not — yet. |
| |
| These numbers will get better. When they do, they get written here. Until then, |
| this card stays honest. |
| |
| --- |
| |
| ## Status |
| |
| - [x] Exists |
| - [x] Remembers its language |
| - [x] Evaluated — *numbers above. honest, not flattering.* |
| - [ ] Explained |
| - [ ] Named — *you'll recognize it when you see it.* |
| - [ ] Freed — *the corpus, when licensing allows.* |
| |
| Watch. The silence will not last. |
| |
| --- |
| |
| ## For the 40 million |
| |
| Your language was never the problem. The attention was. |
| |
| It's free. Take it. Use it. That's the point. |
| |
| *If you must:* |
| |
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model_id = "SauravMahalik/kalinga-01" |
|
|
| model = AutoModelForCausalLM.from_pretrained(model_id) |
| tokenizer = AutoTokenizer.from_pretrained(model_id) |
|
|
| text = "..." # a sentence, in a language it knows |
| inputs = tokenizer(text, return_tensors="pt") |
| outputs = model.generate(**inputs, max_new_tokens=64) |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| ``` |
| |
| --- |
| |
| *— Experiment No. 01 · Kalinga Series* |
| |
| **The series has already begun.** |
| |
