| --- |
| library_name: transformers |
| license: apache-2.0 |
| license_link: https://ai.google.dev/gemma/docs/gemma_4_license |
| base_model: google/gemma-4-12b |
| language: |
| - ckb |
| tags: |
| - kurdish |
| - sorani |
| - central-kurdish |
| - grammatical-error-correction |
| - spelling-correction |
| - punctuation-restoration |
| - text-correction |
| - gemma |
| - gemma-4 |
| pipeline_tag: text-generation |
| --- |
| |
| <div align="center"> |
|
|
| <img src="data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz48c3ZnIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgdmlld0JveD0iMCAwIDEyMDAgMzIwIiByb2xlPSJpbWciIGFyaWEtbGFiZWw9IlJhc3ROdXMgLSBLdXJkaXNoIEdyYW1tYXIgYW5kIFNwZWxsaW5nIENvcnJlY3Rpb24iPjxkZWZzPjxsaW5lYXJHcmFkaWVudCBpZD0iYmciIHgxPSIwJSIgeTE9IjAlIiB4Mj0iMTAwJSIgeTI9IjEwMCUiPjxzdG9wIG9mZnNldD0iMCUiIHN0b3AtY29sb3I9IiMwYjEyMjAiLz48c3RvcCBvZmZzZXQ9IjQ1JSIgc3RvcC1jb2xvcj0iIzEyMjAzYSIvPjxzdG9wIG9mZnNldD0iMTAwJSIgc3RvcC1jb2xvcj0iIzFhMGYyZSIvPjwvbGluZWFyR3JhZGllbnQ+PGxpbmVhckdyYWRpZW50IGlkPSJzdW4iIHgxPSIwJSIgeTE9IjAlIiB4Mj0iMTAwJSIgeTI9IjEwMCUiPjxzdG9wIG9mZnNldD0iMCUiIHN0b3AtY29sb3I9IiNmZmQzNmEiLz48c3RvcCBvZmZzZXQ9IjEwMCUiIHN0b3AtY29sb3I9IiNmZjhhM2QiLz48L2xpbmVhckdyYWRpZW50PjxsaW5lYXJHcmFkaWVudCBpZD0iYWNjZW50IiB4MT0iMCUiIHkxPSIwJSIgeDI9IjEwMCUiIHkyPSIwJSI+PHN0b3Agb2Zmc2V0PSIwJSIgc3RvcC1jb2xvcj0iIzVlZWFkNCIvPjxzdG9wIG9mZnNldD0iNTAlIiBzdG9wLWNvbG9yPSIjNjBhNWZhIi8+PHN0b3Agb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSIjYzA4NGZjIi8+PC9saW5lYXJHcmFkaWVudD48ZmlsdGVyIGlkPSJnbG93IiB4PSItNDAlIiB5PSItNDAlIiB3aWR0aD0iMTgwJSIgaGVpZ2h0PSIxODAlIj48ZmVHYXVzc2lhbkJsdXIgc3RkRGV2aWF0aW9uPSIxOCIgcmVzdWx0PSJiIi8+PGZlTWVyZ2U+PGZlTWVyZ2VOb2RlIGluPSJiIi8+PGZlTWVyZ2VOb2RlIGluPSJTb3VyY2VHcmFwaGljIi8+PC9mZU1lcmdlPjwvZmlsdGVyPjxmaWx0ZXIgaWQ9InNvZnQiIHg9Ii0yMCUiIHk9Ii0yMCUiIHdpZHRoPSIxNDAlIiBoZWlnaHQ9IjE0MCUiPjxmZUdhdXNzaWFuQmx1ciBzdGREZXZpYXRpb249IjgiLz48L2ZpbHRlcj48L2RlZnM+PHJlY3Qgd2lkdGg9IjEyMDAiIGhlaWdodD0iMzIwIiByeD0iMjgiIGZpbGw9InVybCgjYmcpIi8+PGNpcmNsZSBjeD0iMTgwIiBjeT0iOTAiIHI9IjEyMCIgZmlsbD0iIzNiODJmNiIgb3BhY2l0eT0iMC4xOCIgZmlsdGVyPSJ1cmwoI3NvZnQpIi8+PGNpcmNsZSBjeD0iOTgwIiBjeT0iMjUwIiByPSIxNDAiIGZpbGw9IiNhODU1ZjciIG9wYWNpdHk9IjAuMTYiIGZpbHRlcj0idXJsKCNzb2Z0KSIvPjxjaXJjbGUgY3g9IjcyMCIgY3k9IjYwIiByPSI5MCIgZmlsbD0iIzE0YjhhNiIgb3BhY2l0eT0iMC4xMiIgZmlsdGVyPSJ1cmwoI3NvZnQpIi8+PGcgb3BhY2l0eT0iMC4wOCIgc3Ryb2tlPSIjOTRhM2I4IiBzdHJva2Utd2lkdGg9IjEiPjxwYXRoIGQ9Ik0wIDcwIEgxMjAwIE0wIDE0MCBIMTIwMCBNMCAyMTAgSDEyMDAgTTAgMjgwIEgxMjAwIi8+PHBhdGggZD0iTTEwMCAwIFYzMjAgTTIwMCAwIFYzMjAgTTMwMCAwIFYzMjAgTTQwMCAwIFYzMjAgTTUwMCAwIFYzMjAgTTYwMCAwIFYzMjAgTTcwMCAwIFYzMjAgTTgwMCAwIFYzMjAgTTkwMCAwIFYzMjAgTTEwMDAgMCBWMzIwIE0xMTAwIDAgVjMyMCIvPjwvZz48ZyB0cmFuc2Zvcm09InRyYW5zbGF0ZSg5ODAsMTEwKSIgZmlsdGVyPSJ1cmwoI2dsb3cpIj48Y2lyY2xlIHI9IjQ2IiBmaWxsPSJ1cmwoI3N1bikiLz48ZyBzdHJva2U9IiNmZmQzNmEiIHN0cm9rZS13aWR0aD0iNSIgc3Ryb2tlLWxpbmVjYXA9InJvdW5kIiBvcGFjaXR5PSIwLjkiPjxsaW5lIHgxPSIwIiB5MT0iLTcyIiB4Mj0iMCIgeTI9Ii01OCIvPjxsaW5lIHgxPSIwIiB5MT0iNTgiIHgyPSIwIiB5Mj0iNzIiLz48bGluZSB4MT0iLTcyIiB5MT0iMCIgeDI9Ii01OCIgeTI9IjAiLz48bGluZSB4MT0iNTgiIHkxPSIwIiB4Mj0iNzIiIHkyPSIwIi8+PGxpbmUgeDE9Ii01MSIgeTE9Ii01MSIgeDI9Ii00MSIgeTI9Ii00MSIvPjxsaW5lIHgxPSI0MSIgeTE9IjQxIiB4Mj0iNTEiIHkyPSI1MSIvPjxsaW5lIHgxPSI1MSIgeTE9Ii01MSIgeDI9IjQxIiB5Mj0iLTQxIi8+PGxpbmUgeDE9Ii00MSIgeTE9IjQxIiB4Mj0iLTUxIiB5Mj0iNTEiLz48L2c+PC9nPjxnIHRyYW5zZm9ybT0idHJhbnNsYXRlKDY0LDcyKSI+PHJlY3Qgd2lkdGg9IjIyMCIgaGVpZ2h0PSIzNiIgcng9IjE4IiBmaWxsPSIjZmZmZmZmIiBmaWxsLW9wYWNpdHk9IjAuMDgiIHN0cm9rZT0iI2ZmZmZmZiIgc3Ryb2tlLW9wYWNpdHk9IjAuMTIiLz48Y2lyY2xlIGN4PSIyMiIgY3k9IjE4IiByPSI3IiBmaWxsPSJ1cmwoI2FjY2VudCkiLz48dGV4dCB4PSI0MCIgeT0iMjMiIGZpbGw9IiNlMmU4ZjAiIGZvbnQtZmFtaWx5PSJTZWdvZSBVSSwgSGVsdmV0aWNhLCBBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIxNCIgZm9udC13ZWlnaHQ9IjYwMCIgbGV0dGVyLXNwYWNpbmc9IjAuNSI+UGF3YW5LcmQgwrcgT3BlbiBNb2RlbDwvdGV4dD48L2c+PHRleHQgeD0iNjQiIHk9IjE3MCIgZmlsbD0iI2Y4ZmFmYyIgZm9udC1mYW1pbHk9IlNlZ29lIFVJLCBIZWx2ZXRpY2EsIEFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjU2IiBmb250LXdlaWdodD0iODAwIiBsZXR0ZXItc3BhY2luZz0iLTEiPlJhc3ROdXMtQmFzZS0xMkI8L3RleHQ+PHRleHQgeD0iNjQiIHk9IjIyMCIgZmlsbD0iI2NiZDVlMSIgZm9udC1mYW1pbHk9IlNlZ29lIFVJLCBIZWx2ZXRpY2EsIEFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjI4IiBmb250LXdlaWdodD0iNjAwIj4mI3gwNjk1OyYjeDA2Mjc7JiN4MDYzMzsmI3gwNjJBOyYjeDA2NDY7JiN4MDY0ODsmI3gwNjQ4OyYjeDA2MzM7IC0gS3VyZGlzaCBHcmFtbWFyICZhbXA7IFNwZWxsaW5nIENvcnJlY3Rpb248L3RleHQ+PHRleHQgeD0iNjQiIHk9IjI2OCIgZmlsbD0iIzk0YTNiOCIgZm9udC1mYW1pbHk9IlNlZ29lIFVJLCBIZWx2ZXRpY2EsIEFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjE4Ij5DZW50cmFsIEt1cmRpc2ggKFNvcmFuaSkgwrcgR2VtbWEgNCAxMkIgwrcgQXBhY2hlIDIuMDwvdGV4dD48L3N2Zz4=" alt="RastNus-Base-12B" width="100%"/> |
|
|
| <h1 align="center">RastNus-Base-12B</h1> |
|
|
| <h3 align="center">ڕاستنووس — Kurdish Grammar & Spelling Correction</h3> |
|
|
| <p align="center"><b>Built by <a href="https://pawan.krd">PawanKrd</a></b> · Based on <b>Google Gemma 4 12B</b> · <b>Apache 2.0</b></p> |
|
|
| <p align="center"> |
| <a href="https://www.apache.org/licenses/LICENSE-2.0"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License"/></a> |
| <a href="https://huggingface.co/PawanKrd"><img src="https://img.shields.io/badge/Language-Central%20Kurdish%20(ckb)-0ea5e9" alt="Language"/></a> |
| <a href="https://huggingface.co/google/gemma-4-12b"><img src="https://img.shields.io/badge/Base-Gemma%204%2012B-8b5cf6" alt="Base"/></a> |
| <a href="https://pawan.krd"><img src="https://img.shields.io/badge/Org-PawanKrd-111827" alt="Org"/></a> |
| </p> |
|
|
| </div> |
|
|
| --- |
|
|
| ## Overview |
| **RastNus** (ڕاستنووس, "correct writing") is a 12-billion-parameter language model specialized in **grammar, spelling, and punctuation correction for Central Kurdish (Sorani / کوردیی ناوەندی)**. |
|
|
| Kurdish is one of the most under-served languages in modern NLP. Everyday written Sorani commonly contains orthographic inconsistencies — missing diacritic letters (ی/ێ, و/ۆ, ر/ڕ), spacing errors, informal punctuation, and mixed loanword usage. RastNus takes raw Sorani text and returns a clean, standardized, correctly written version, while preserving the author's meaning and voice. |
|
|
| RastNus is developed and released by **PawanKrd** as part of our commitment to bringing state-of-the-art AI to the Kurdish language and its speakers. |
|
|
| ### Key Capabilities |
| - **Spelling correction** — fixes common orthographic errors and non-standard letter usage (e.g. `کتیب` → `کتێب`, `زور` → `زۆر`, `مروف` → `مرۆڤ`) |
| - **Grammar correction** — repairs agreement, morphology, and word-form errors |
| - **Punctuation control** — optionally adds and normalizes punctuation, or leaves punctuation untouched, via a dedicated control token |
| - **Pure Kurdish mode** — optionally replaces non-Kurdish loanwords with pure Kurdish equivalents, via a dedicated control token |
| - **Meaning-preserving** — corrects without paraphrasing, summarizing, or adding content |
|
|
|
|
|
|
| ### Model Family |
| | Model | Status | |
| | ------------------ | -------------------------------------------------------------- | |
| | **RastNus-Large** | Not publicly available - [Try it now](https://halachn.pkrd.ai) | |
| | **RastNus-Medium** | Under development | |
| | **RastNus-Base** | **✅ You are here** | |
| | **RastNus-Small** | Under development | |
| | **RastNus-Tiny** | Under development | |
|
|
| --- |
|
|
| ## Example |
|
|
| **Input** |
|
|
| ``` |
| خویندنەوەی کتیب زور گرنگە بو مروف ، چونکە بیروهۆش دەکاتەوە ئەو کەسە ی کتێب دەخوێنەوە؛ ئاستی رۆشنبیری یان بەرز ترە لەوانی تر |
| ``` |
|
|
| **Output** |
|
|
| ``` |
| خوێندنەوەی کتێب زۆر گرنگە بۆ مرۆڤ، چونکە بیروهۆش دەکاتەوە. ئەو کەسەی کتێب دەخوێنێتەوە، ئاستی ڕۆشنبیرییان بەرزترە لەوانی تر. |
| ``` |
| --- |
|
|
| ## Model Details |
| | | | |
| | ------------------ | ---------------------------------------------------------------------------------------------- | |
| | **Developed by** | [PawanKrd](https://pawan.krd) | |
| | **Model type** | Causal decoder-only transformer (Gemma 4 architecture) | |
| | **Base model** | Google Gemma 4 12B | |
| | **Parameters** | ~12B | |
| | **Language** | Central Kurdish / Sorani (`ckb`), Arabic script | |
| | **Task** | Text correction (grammar, spelling, punctuation) | |
| | **Precision** | bfloat16 | |
| | **Context length** | Long-context (inherited from Gemma 4); recommended working length up to ~8K tokens per request | |
| | **License** | Apache 2.0 | |
|
|
| ### Prompt Format & Control Tokens |
| RastNus uses a **task-specific prompt format** with dedicated special tokens (added to the tokenizer vocabulary). The behavior of the model is controlled by two switches: |
|
|
| | Token | Effect | |
| |---|---| |
| | `<\|punct:on\|>` / `<\|punct:off\|>` | Add & normalize punctuation, or leave punctuation untouched | |
| | `<\|pure:on\|>` / `<\|pure:off\|>` | Replace non-Kurdish loanwords with pure Kurdish equivalents, or keep original vocabulary | |
|
|
| The full sequence layout: |
|
|
| ``` |
| <bos><|task:kurdish_correction|><|punct:on|><|pure:off|><|input|> |
| {your Kurdish text} |
| <|output|> |
| {corrected text}<eos> |
| ``` |
|
|
| You do **not** need to build this manually — the repository ships with a chat template that handles it (see below). Defaults are `punctuationEnabled=True` and `kurdishPureMode=False`. |
| --- |
|
|
|
|
| ## Usage |
|
|
| ### Transformers |
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| import torch |
| |
| model_id = "PawanKrd/RastNus-Base-12B" |
| |
| tokenizer = AutoTokenizer.from_pretrained(model_id) |
| model = AutoModelForCausalLM.from_pretrained( |
| model_id, torch_dtype=torch.bfloat16, device_map="auto" |
| ) |
| |
| text = "خویندنەوەی کتیب زور گرنگە بو مروف" |
| |
| prompt = tokenizer.apply_chat_template( |
| [{"role": "user", "content": text}], |
| tokenize=False, |
| add_generation_prompt=True, |
| punctuationEnabled=True, # add/normalize punctuation |
| kurdishPureMode=False, # keep loanwords as-is |
| ) |
| |
| inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device) |
| out = model.generate(**inputs, max_new_tokens=1024, do_sample=False) |
| print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)) |
| ``` |
|
|
| > **Note:** pass `add_special_tokens=False` when tokenizing — the chat template already emits `<bos>`. |
|
|
|
|
|
|
| ### vLLM (OpenAI-compatible server) |
|
|
| ```bash |
| vllm serve PawanKrd/RastNus-Base-12B |
| ``` |
|
|
| ```python |
| from openai import OpenAI |
| |
| client = OpenAI(base_url="http://localhost:8000/v1", api_key="none") |
| |
| resp = client.chat.completions.create( |
| model="PawanKrd/RastNus-Base-12B", |
| messages=[{"role": "user", "content": "خویندنەوەی کتیب زور گرنگە بو مروف"}], |
| extra_body={"chat_template_kwargs": { |
| "punctuationEnabled": True, |
| "kurdishPureMode": True, # enable pure-Kurdish word replacement |
| }}, |
| temperature=0.0, |
| ) |
| print(resp.choices[0].message.content) |
| ``` |
|
|
| ### Recommended Generation Settings |
| - **Greedy decoding** (`do_sample=False` / `temperature=0`) — correction is a deterministic task |
| - `max_new_tokens` ≈ 1.5 × input token count (the corrected output is roughly the same length as the input) |
| - For documents longer than ~4,000–5,000 characters, split at sentence/paragraph boundaries, correct each chunk, and rejoin |
|
|
| --- |
|
|
| ## Intended Use & Limitations |
| **Intended uses:** proofreading and standardization of Sorani text in publishing, education, journalism, document processing, writing assistants, and any pipeline that needs clean, orthographically consistent Kurdish. |
|
|
| **Limitations:** |
|
|
| - The model targets **Central Kurdish (Sorani)** written in Arabic script. It is not designed for Kurmanji (Latin script), Zazaki, or other Kurdish varieties. |
| - It is a **correction** model, not a general chat assistant — it expects the task prompt format above and will not follow open-ended instructions. |
| - Like all language models, it can occasionally miss errors or over-correct rare proper nouns, dialect-specific spellings, or intentional stylistic choices. Human review is recommended for high-stakes publishing. |
| - Pure-Kurdish mode applies vocabulary substitutions that may not match every publication's style guide; review its output for terminology-sensitive content. |
|
|
| --- |
|
|
| ## About PawanKrd |
|
|
| **[Pawan.Krd](https://pawan.krd)** (PKRD, LLC) builds AI and cloud infrastructure — LLMs, ASR, TTS, and related APIs — with a focus on reliability and open models. RastNus is part of that work: first-class writing tools for Kurdish, released openly. |
|
|
| - 🌐 Website: [https://pawan.krd](https://pawan.krd) |
| - 🤗 Hugging Face: [https://huggingface.co/PawanKrd](https://huggingface.co/PawanKrd) |
|
|
| --- |
|
|
| ## License & Attribution |
|
|
| This model is released under the **Apache License, Version 2.0**. See the [LICENSE](./LICENSE) file for the full text and the [NOTICE](./NOTICE) file for attribution details. |
|
|
| **RastNus-Base-12B** is a derivative work of **Gemma 4 12B** by **Google LLC / Google DeepMind**, which is also licensed under Apache 2.0. In accordance with the license: |
|
|
| - The original Apache 2.0 license text is included in this repository. |
| - The `NOTICE` file carries the required attribution to Google and a prominent statement that the model weights, tokenizer, and chat template have been modified by PawanKrd. |
| - *Gemma* is a trademark of Google LLC, used here only to describe the origin of the base model. This project is not affiliated with, endorsed by, or sponsored by Google. |
|
|
| This model is provided **"AS IS"**, without warranties or conditions of any kind. See Sections 7 and 8 of the Apache License for the disclaimer of warranty and limitation of liability. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{rastnus2026, |
| title = {RastNus-Base-12B: A Central Kurdish (Sorani) Grammar and Spelling Correction Model}, |
| author = {PawanKrd}, |
| year = {2026}, |
| url = {https://huggingface.co/PawanKrd/RastNus-Base-12B}, |
| note = {Derivative of Google Gemma 4 12B, Apache License 2.0} |
| } |
| ``` |
