| # ak-lm-small |
|
|
| **Tiny pretrained Malayalam language model (75M parameters)** |
|
|
| AK-LM-SMALL (Akshara Language Model Small) is a 75M small Malayalam GPT style Model ,Build from Scratch |
|
|
| Developed by **EnduraSolution**. |
|
|
| ## Overview |
|
|
| - GPT-style decoder-only Transformer |
| - 75 Million parameters |
| - Custom Malayalam SentencePiece tokenizer |
| - Vocabulary: 24,000 |
| - Context length: 256 |
|
|
| ## Training Data |
|
|
| - IndicCorp Malayalam |
| - Malayalam Wikipedia |
| - Kerala-focused corpus |
| - Small English Wikipedia subset |
|
|
| ## Intended Uses |
|
|
| - Malayalam text generation |
| - Text completion |
| - Research |
| - Fine-tuning |
|
|
| ## Limitations |
|
|
| This is a pretrained base model. |
|
|
| It is **not instruction tuned**. |
|
|
| ## Example |
|
|
| ```python |
| from transformers import GPT2LMHeadModel |
| import sentencepiece as spm |
| |
| sp = spm.SentencePieceProcessor(model_file='ml_sp.model') |
| model = GPT2LMHeadModel.from_pretrained('endurasolution/ak-lm-small') |
| ``` |
|
|
| ## License |
|
|
| Apache-2.0 |
|
|
| --- |
|
|
| Made with ❤️ by EnduraSolution |
