# Custom Urdu LLM

This is a custom transformer-based Large Language Model for Urdu.

## Model Details
- **Architecture:** Transformer (GPT-based)
- **Framework:** PyTorch
- **Tokenizer:** SentencePiece
- **Hyperparameters:**
  - Vocabulary Size: 20,000
  - Embedding Size: 768
  - Attention Heads: 12
  - Layers: 12
  - Dropout: 0.2

## Usage

```python
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("AliMuhammad73/testing-model")
tokenizer = AutoTokenizer.from_pretrained("AliMuhammad73/testing-model")

prompt = <prompt in urdu>
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(inputs.input_ids, max_new_tokens=tokens_to_generate)
print(tokenizer.decode(output[0]))
```

---
license: apache-2.0
---