---
library_name: transformers
pipeline_tag: text-generation
tags:
- BGPT
- meta
- pytorch
- llama
- llama-3
---


# Model Information

BGPT is a finetuned version of Llama3.2-3B-Instruct, specifically optimized for generating high-quality multilingual outputs across 11 Indic languages. The model demonstrates strong capabilities in translation, summarization, and conversational tasks while maintaining the base model's performance characteristics.

## Model Developer
Harsh Bande

## Model Architecture
- **Base Model:** Llama3.2-3B-Instruct
- **Model type:** Finetuned LLaMA (Language Model for Multilingual Text Generation)
- **Architecture Type:** Auto-regressive language model with optimized transformer architecture
- **Adaptation Method:** LoRA (Low-Rank Adaptation)
- **Model Type:** Instruction-tuned multilingual text generation model

## Supported Languages
Hindi, Punjabi, Marathi, Malayalam, Oriya, Kannada, Gujarati, Bengali, Urdu, Tamil, and Telugu

# Intended Use

## Primary Use Cases
- Multilingual text generation
- Cross-lingual translation
- Text summarization
- Conversational AI in Indic languages
- Language understanding and generation tasks

## How to Get Started with the Model

Make sure to update your transformers installation via `pip install --upgrade transformers`.

Use the code below to get started with the model.

```python
import torch
from transformers import pipeline

model_id = "Onkarn/ML-Test-v01"
pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
messages = [
    {"role": "system", "content": "You are a helpful assistant who responds in hindi"},
    {"role": "user", "content": "कर्नाटक की राजधानी क्या है?"},
]
outputs = pipe(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])
```


## Training Details

### Training Data

- **Dataset Composition:** Curated collection of text from 11 Indic languages
- **Languages Covered:** Hindi, Punjabi, Marathi, Malayalam, Oriya, Kannada, Gujarati, Bengali, Urdu, Tamil, and Telugu

### Training Parameters

- **Optimization Technique**: LoRA (Low-Rank Adaptation)
- **Epochs**: 3.0
- **Batch Size**: 2.0 (per device train batch size)
- **Learning Rate**: 5e-05


## Hardware and Environmental Impact

### Training Infrastructure
- **Hardware:** T4 GPU
- **Cloud Provider:** Google Cloud Platform
- **Region:** asia-southeast1
- **Training Duration:** 29 hours

### Environmental Impact Assessment
- **Carbon Emissions:** 0.85 kgCO₂eq
- **Carbon Offset:** 100% offset by the cloud provider
- **Location:** asia-southeast1 region

## Limitations and Biases
- The model's performance may vary across different Indic languages
- The model inherits both capabilities and limitations of the base Llama3.2-3B-Instruct model
- Users should conduct appropriate testing for their specific use cases

## License
[More Information Needed]

## Citation and References
[More Information Needed]