| # Custom Urdu LLM | |
| This is a custom transformer-based Large Language Model for Urdu. | |
| ## Model Details | |
| - **Architecture:** Transformer (GPT-based) | |
| - **Framework:** PyTorch | |
| - **Tokenizer:** SentencePiece | |
| - **Hyperparameters:** | |
| - Vocabulary Size: 20,000 | |
| - Embedding Size: 768 | |
| - Attention Heads: 12 | |
| - Layers: 12 | |
| - Dropout: 0.2 | |
| ## Usage | |
| First you will need to download the ```modeling_gpt.py``` file from the repo. Once that's been done, you can define another file and use the following code to generate text from the model: | |
| ```python | |
| from modeling_gpt import GPTLanguageModel | |
| from transformers import AutoTokenizer | |
| model = GPTLanguageModel.from_pretrained("AliMuhammad73/testing-model") | |
| tokenizer = AutoTokenizer.from_pretrained("AliMuhammad73/testing-model") | |
| # sentence in urdu | |
| prompt = "پاکستان ایک ایسا ملک ہے جو جنوبی ایشیا میں واقع ہے۔ اس کی سرحدیں ہندوستان، چین، افغانستان، اور " | |
| encoded = tokenizer.encode(prompt) | |
| encoded_tensor = torch.tensor(encoded).unsqueeze(0) | |
| output = model.generate(encoded_tensor, max_new_tokens=64) | |
| response = tokenizer.decode(output[0].squeeze().tolist()) | |
| ``` | |
| --- | |
| license: apache-2.0 | |
| --- |