README.md

---
license: apache-2.0
language:
- dv
base_model: Qwen/Qwen3-8B
tags:
- dhivehi
- maldives
- thaana
- qwen3
- lora
- instruction-tuned
library_name: transformers
pipeline_tag: text-generation
extra_gated_prompt: >-
  Access to Naturecode Dhivehi requires approval. This model is intended for
  research and development purposes for the Dhivehi language community. Please
  provide your intended use case for review.
extra_gated_fields:
  Name: text
  Organization: text
  Country: text
  Use Case: text
  I agree to use this model responsibly: checkbox
---

# Naturecode Dhivehi

**The first production-ready Dhivehi language model for the Maldives.**

Naturecode Dhivehi is a fine-tuned version of Qwen3-8B, trained specifically for the Dhivehi language (ދިވެހި) with comprehensive instruction-following capabilities.

## Model Details

| Attribute | Value |
|-----------|-------|
| **Base Model** | Qwen/Qwen3-8B |
| **Training Method** | CPT + SFT with LoRA |
| **LoRA Rank** | 64 |
| **LoRA Alpha** | 128 |
| **Languages** | Dhivehi (ދިވެހި), English |

## Capabilities

- **Formal Writing**: Letters, proposals, applications, official documents
- **Informal Dhivehi**: Chat, social media, texting (romanized & Thaana)
- **Creative Writing**: Stories, poems, songs, Boduberu lyrics
- **Cultural Knowledge**: Maldivian traditions, customs, Islamic practices
- **Translation**: English to Dhivehi bidirectional
- **Q&A**: General knowledge about Maldives, geography, history
- **Technical Writing**: Reports, documentation, explanations

## Important: System Prompt

For best results, **always use this system prompt**:

```
ތިބާއަކީ ދިވެހި ބަހުގެ އެހީތެރިއެކެވެ. ކޮންމެ ޖަވާބެއްގައި މަދުވެގެން 5-8 ޖުމްލަ ހިމެނެން ވާނެއެވެ. ތަފްސީލީ، ފުރިހަމަ ޖަވާބުތައް ދެއްވާށެވެ.
```

Translation: "You are a Dhivehi language assistant. Each response should contain at least 5-8 sentences. Provide detailed, complete answers."

## Usage

### With ms-swift (Recommended)

```bash
pip install ms-swift

swift infer \
    --model Qwen/Qwen3-8B \
    --adapters Naturecode/dhivehi \
    --stream true
```

### With Transformers + PEFT

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-8B",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")

# Load adapter
model = PeftModel.from_pretrained(base_model, "Naturecode/dhivehi")

# System prompt
system = "ތިބާއަކީ ދިވެހި ބަހުގެ އެހީތެރިއެކެވެ. ކޮންމެ ޖަވާބެއްގައި މަދުވެގެން 5-8 ޖުމްލަ ހިމެނެން ވާނެއެވެ. ތަފްސީލީ، ފުރިހަމަ ޖަވާބުތައް ދެއްވާށެވެ."

messages = [
    {"role": "system", "content": system},
    {"role": "user", "content": "ދިވެހިރާއްޖޭގެ ވެރިރަށަކީ ކޮބާ؟"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Example Output

**Question:** ދިވެހިރާއްޖޭގެ ވެރިރަށަކީ ކޮބާ؟

**Response:** ދިވެހިރާއްޖޭގެ ވެރިރަށަކީ މާލެއެވެ. މާލެއަކީ ދިވެހިރާއްޖޭގެ އެންމެ ބޮޑު ސިޓީ އަދި ސިޔާސީ، އިޤްތިޞާދީ، ސަޤާފީ މަރުކަޒެވެ. މި ރަށުގައި ގާތްގަނޑަކަށް 200,000 އެއްހާ މީހުން ދިރިއުޅެއެވެ. މާލެއަކީ ދުނިޔޭގެ އެންމެ ތޮއްޖެހިފައިވާ ސިޓީތަކުގެ ތެރެއިން ސިޓީއެކެވެ.

## Maldives Knowledge

| Fact | Value |
|------|-------|
| Capital | މާލެ (Malé) |
| Islands | ~1,200 total, ~200 inhabited |
| Population | ~500,000 |
| Currency | Rufiyaa (ރުފިޔާ / MVR) |
| Religion | Islam (100%) |
| Language | Dhivehi (ދިވެހި) |
| Independence | July 26, 1965 |

## Training

Two-stage training approach:

1. **Continued Pretraining (CPT)**: Extended pretraining on Dhivehi text
2. **Supervised Fine-Tuning (SFT)**: Instruction tuning on curated Dhivehi data

## Limitations

- Best performance with the recommended system prompt
- May occasionally mix English words in responses
- Knowledge cutoff applies to recent events

## Intended Use

- Dhivehi language research and development
- Building Dhivehi-language applications
- Educational tools for the Maldivian community
- Translation assistance

## License

Apache 2.0

## Citation

```bibtex
@misc{naturecode-dhivehi,
  title={Naturecode Dhivehi: A Production-Ready Dhivehi Language Model},
  author={Naturecode},
  year={2024},
  publisher={HuggingFace},
  url={https://huggingface.co/Naturecode/dhivehi}
}
```

---

**Built for the Maldives**