File size: 3,016 Bytes
2266759 2832d11 5aca7df 2266759 5aca7df 2266759 2832d11 537544f 2832d11 537544f 2832d11 537544f 5aca7df 537544f 2832d11 5aca7df 31b0b6a 5aca7df 31b0b6a efc0a4f 31b0b6a efc0a4f 31b0b6a 2832d11 537544f 2832d11 5aca7df 2832d11 5aca7df 2832d11 537544f 2832d11 537544f 2832d11 537544f 5aca7df | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 | ---
library_name: transformers
pipeline_tag: text-generation
tags:
- BGPT
- meta
- pytorch
- llama
- llama-3
---
# Model Information
BGPT is a finetuned version of Llama3.2-3B-Instruct, specifically optimized for generating high-quality multilingual outputs across 11 Indic languages. The model demonstrates strong capabilities in translation, summarization, and conversational tasks while maintaining the base model's performance characteristics.
## Model Developer
Harsh Bande
## Model Architecture
- **Base Model:** Llama3.2-3B-Instruct
- **Model type:** Finetuned LLaMA (Language Model for Multilingual Text Generation)
- **Architecture Type:** Auto-regressive language model with optimized transformer architecture
- **Adaptation Method:** LoRA (Low-Rank Adaptation)
- **Model Type:** Instruction-tuned multilingual text generation model
## Supported Languages
Hindi, Punjabi, Marathi, Malayalam, Oriya, Kannada, Gujarati, Bengali, Urdu, Tamil, and Telugu
# Intended Use
## Primary Use Cases
- Multilingual text generation
- Cross-lingual translation
- Text summarization
- Conversational AI in Indic languages
- Language understanding and generation tasks
## How to Get Started with the Model
Make sure to update your transformers installation via `pip install --upgrade transformers`.
Use the code below to get started with the model.
```python
import torch
from transformers import pipeline
model_id = "Onkarn/ML-Test-v01"
pipe = pipeline(
"text-generation",
model=model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
messages = [
{"role": "system", "content": "You are a helpful assistant who responds in hindi"},
{"role": "user", "content": "कर्नाटक की राजधानी क्या है?"},
]
outputs = pipe(
messages,
max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])
```
## Training Details
### Training Data
- **Dataset Composition:** Curated collection of text from 11 Indic languages
- **Languages Covered:** Hindi, Punjabi, Marathi, Malayalam, Oriya, Kannada, Gujarati, Bengali, Urdu, Tamil, and Telugu
### Training Parameters
- **Optimization Technique**: LoRA (Low-Rank Adaptation)
- **Epochs**: 3.0
- **Batch Size**: 2.0 (per device train batch size)
- **Learning Rate**: 5e-05
## Hardware and Environmental Impact
### Training Infrastructure
- **Hardware:** T4 GPU
- **Cloud Provider:** Google Cloud Platform
- **Region:** asia-southeast1
- **Training Duration:** 29 hours
### Environmental Impact Assessment
- **Carbon Emissions:** 0.85 kgCO₂eq
- **Carbon Offset:** 100% offset by the cloud provider
- **Location:** asia-southeast1 region
## Limitations and Biases
- The model's performance may vary across different Indic languages
- The model inherits both capabilities and limitations of the base Llama3.2-3B-Instruct model
- Users should conduct appropriate testing for their specific use cases
## License
[More Information Needed]
## Citation and References
[More Information Needed]
|