File size: 1,213 Bytes
1fab0d9 dbee900 1fab0d9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
# Custom Urdu LLM
This is a custom transformer-based Large Language Model for Urdu.
## Model Details
- **Architecture:** Transformer (GPT-based)
- **Framework:** PyTorch
- **Tokenizer:** SentencePiece
- **Hyperparameters:**
- Vocabulary Size: 20,000
- Embedding Size: 768
- Attention Heads: 12
- Layers: 12
- Dropout: 0.2
## Usage
First you will need to download the ```modeling_gpt.py``` file from the repo. Once that's been done, you can define another file and use the following code to generate text from the model:
```python
from modeling_gpt import GPTLanguageModel
from transformers import AutoTokenizer
model = GPTLanguageModel.from_pretrained("AliMuhammad73/testing-model")
tokenizer = AutoTokenizer.from_pretrained("AliMuhammad73/testing-model")
# sentence in urdu
prompt = "پاکستان ایک ایسا ملک ہے جو جنوبی ایشیا میں واقع ہے۔ اس کی سرحدیں ہندوستان، چین، افغانستان، اور "
encoded = tokenizer.encode(prompt)
encoded_tensor = torch.tensor(encoded).unsqueeze(0)
output = model.generate(encoded_tensor, max_new_tokens=64)
response = tokenizer.decode(output[0].squeeze().tolist())
```
---
license: apache-2.0
--- |