| --- |
| language: |
| - so |
| - en |
|
|
| tags: |
| - text-generation |
| - causal-lm |
|
|
| license: gemma |
| base_model: google/gemma-3-270m |
|
|
| datasets: |
| - maanka2/somali-web-corpus |
|
|
| metrics: |
| - loss |
| --- |
| |
| # SOMGPT |
|
|
| somgpt-base is a Somali causal language model continued from google/gemma-3-270m and trained on maanka2/somali-web-corpus. |
|
|
| ## Model Details |
|
|
| - Developer: maanka2 |
| - Architecture: Gemma 3 (270M) |
| - Model Type: Causal Language Model |
| - Language: Somali |
| - Base Model: google/gemma-3-270m |
| - Dataset: maanka2/somali-web-corpus |
| - License: gemma |
|
|
| ## Overview |
|
|
| This model was further pre-trained on Somali web text to improve its understanding of Somali vocabulary, grammar, spelling, and writing patterns. |
|
|
| somgpt is a base language model designed for text continuation and language modeling. It is not instruction-tuned and is not optimized for chat, question answering, or assistant-style interactions. |
|
|
| For conversational AI or task-specific applications, additional supervised fine-tuning (SFT) or instruction tuning is recommended. |
|
|
| ## Training Data |
|
|
| Training was performed using maanka2/somali-web-corpus, a collection of cleaned Somali-language web content gathered from various online sources. |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| |
| model_id = "maanka2/somgpt" |
| |
| tokenizer = AutoTokenizer.from_pretrained(model_id) |
| model = AutoModelForCausalLM.from_pretrained(model_id) |
| |
| prompt = "Soomaaliya waa dal ku yaal" |
| inputs = tokenizer(prompt, return_tensors="pt") |
| |
| outputs = model.generate( |
| **inputs, |
| max_new_tokens=256, |
| do_sample=True, |
| temperature=0.1, |
| top_p=0.95 |
| ) |
| |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
