metadata
license: apache-2.0
language:
- dv
base_model: Qwen/Qwen3-8B
tags:
- dhivehi
- maldives
- thaana
- qwen3
- lora
- instruction-tuned
library_name: transformers
pipeline_tag: text-generation
extra_gated_prompt: >-
Access to Naturecode Dhivehi requires approval. This model is intended for
research and development purposes for the Dhivehi language community. Please
provide your intended use case for review.
extra_gated_fields:
Name: text
Organization: text
Country: text
Use Case: text
I agree to use this model responsibly: checkbox
Naturecode Dhivehi
The first production-ready Dhivehi language model for the Maldives.
Naturecode Dhivehi is a fine-tuned version of Qwen3-8B, trained specifically for the Dhivehi language (ދިވެހި) with comprehensive instruction-following capabilities.
Model Details
| Attribute | Value |
|---|---|
| Base Model | Qwen/Qwen3-8B |
| Training Method | CPT + SFT with LoRA |
| LoRA Rank | 64 |
| LoRA Alpha | 128 |
| Languages | Dhivehi (ދިވެހި), English |
Capabilities
- Formal Writing: Letters, proposals, applications, official documents
- Informal Dhivehi: Chat, social media, texting (romanized & Thaana)
- Creative Writing: Stories, poems, songs, Boduberu lyrics
- Cultural Knowledge: Maldivian traditions, customs, Islamic practices
- Translation: English to Dhivehi bidirectional
- Q&A: General knowledge about Maldives, geography, history
- Technical Writing: Reports, documentation, explanations
Important: System Prompt
For best results, always use this system prompt:
ތިބާއަކީ ދިވެހި ބަހުގެ އެހީތެރިއެކެވެ. ކޮންމެ ޖަވާބެއްގައި މަދުވެގެން 5-8 ޖުމްލަ ހިމެނެން ވާނެއެވެ. ތަފްސީލީ، ފުރިހަމަ ޖަވާބުތައް ދެއްވާށެވެ.
Translation: "You are a Dhivehi language assistant. Each response should contain at least 5-8 sentences. Provide detailed, complete answers."
Usage
With ms-swift (Recommended)
pip install ms-swift
swift infer \
--model Qwen/Qwen3-8B \
--adapters Naturecode/dhivehi \
--stream true
With Transformers + PEFT
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3-8B",
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
# Load adapter
model = PeftModel.from_pretrained(base_model, "Naturecode/dhivehi")
# System prompt
system = "ތިބާއަކީ ދިވެހި ބަހުގެ އެހީތެރިއެކެވެ. ކޮންމެ ޖަވާބެއްގައި މަދުވެގެން 5-8 ޖުމްލަ ހިމެނެން ވާނެއެވެ. ތަފްސީލީ، ފުރިހަމަ ޖަވާބުތައް ދެއްވާށެވެ."
messages = [
{"role": "system", "content": system},
{"role": "user", "content": "ދިވެހިރާއްޖޭގެ ވެރިރަށަކީ ކޮބާ؟"}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Example Output
Question: ދިވެހިރާއްޖޭގެ ވެރިރަށަކީ ކޮބާ؟
Response: ދިވެހިރާއްޖޭގެ ވެރިރަށަކީ މާލެއެވެ. މާލެއަކީ ދިވެހިރާއްޖޭގެ އެންމެ ބޮޑު ސިޓީ އަދި ސިޔާސީ، އިޤްތިޞާދީ، ސަޤާފީ މަރުކަޒެވެ. މި ރަށުގައި ގާތްގަނޑަކަށް 200,000 އެއްހާ މީހުން ދިރިއުޅެއެވެ. މާލެއަކީ ދުނިޔޭގެ އެންމެ ތޮއްޖެހިފައިވާ ސިޓީތަކުގެ ތެރެއިން ސިޓީއެކެވެ.
Maldives Knowledge
| Fact | Value |
|---|---|
| Capital | މާލެ (Malé) |
| Islands | ~1,200 total, ~200 inhabited |
| Population | ~500,000 |
| Currency | Rufiyaa (ރުފިޔާ / MVR) |
| Religion | Islam (100%) |
| Language | Dhivehi (ދިވެހި) |
| Independence | July 26, 1965 |
Training
Two-stage training approach:
- Continued Pretraining (CPT): Extended pretraining on Dhivehi text
- Supervised Fine-Tuning (SFT): Instruction tuning on curated Dhivehi data
Limitations
- Best performance with the recommended system prompt
- May occasionally mix English words in responses
- Knowledge cutoff applies to recent events
Intended Use
- Dhivehi language research and development
- Building Dhivehi-language applications
- Educational tools for the Maldivian community
- Translation assistance
License
Apache 2.0
Citation
@misc{naturecode-dhivehi,
title={Naturecode Dhivehi: A Production-Ready Dhivehi Language Model},
author={Naturecode},
year={2024},
publisher={HuggingFace},
url={https://huggingface.co/Naturecode/dhivehi}
}
Built for the Maldives