Create README.md

README.md

@@ -0,0 +1,60 @@
+---
+license: mit
+language:
+- si
+base_model:
+- HuggingFaceTB/SmolLM2-1.7B
+library_name: transformers
+tags:
+- Genral
+- text-generation-inference
+---
+# SinhalaLLM (Fine-tuned SmolLM2 + Sinhala tokenizer)
+
+Model: HuggingFaceTB/SmolLM2-1.7B (base) + LoRA finetune (merged)
+Tokenizer: polyglots/Extended-Sinhala-LLaMA (custom Sinhala tokenizer)
+Language: Sinhala (si)
+
+## Summary
+This model is a SmolLM2-1.7B base model fine-tuned on Sinhala text (MADLAD_CulturaX_cleaned).
+Finetuning method: 4-bit LoRA finetuning via Unsloth + PEFT; final artifact merged into a standard HF model.
+
+## Training data
+- Source: polyglots/MADLAD_CulturaX_cleaned (filtered to `lang == "si"`)
+- Preprocessing: cleaned and deduplicated; chunked into sequences of length 256; tokenized with `polyglots/Extended-Sinhala-LLaMA`.
+- Train/validation split: 99% / 1%.
+
+## Hyperparameters (high-level)
+- Sequence length: 256
+- LoRA rank (r): 16
+- LoRA alpha: 16
+- LoRA dropout: 0.05
+- Optimizer: AdamW fused
+- Learning rate: 2e-4
+- Batch size (effective): per-device batch 8, gradient accumulation 2 (effective 16)
+- Mixed precision: bf16 or fp16 where available
+
+## Evaluation
+- Quick evaluation performed on a held-out 1% validation sample,
+- Reported metric: perplexity (see run logs in the repo)
+
+## How to use
+Install transformers and load:
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+tok = AutoTokenizer.from_pretrained("path_or_repo/sinhala_merged")
+model = AutoModelForCausalLM.from_pretrained("path_or_repo/sinhala_merged", device_map="auto")
+````
+
+## Export / Run locally
+
+* To run on CPU or inference frameworks you can create a GGUF with `llama.cpp` converters and quantize to Q4 variants.
+
+## Limitations and risks
+
+* Model trained on web-scraped data; it may reproduce harmful content or biases present in the training data.
+* Not safe for high-stakes medical, legal, or safety-critical advice.
+
+## License
+
+Specify dataset and model license here.