# 🧠 Myanmar LLM Training

Fine-tune **Qwen2.5-0.5B-Instruct** with Myanmar language dataset.

## ⚡ No License Required!

This model is fully open. No Llama license needed!

## 📋 Requirements

- Python 3.8+
- GPU with 6GB+ VRAM
- HuggingFace Account

## 🚀 Quick Start

### 1. Install dependencies
```bash
pip install -r requirements.txt
```

### 2. Login to HuggingFace
```bash
huggingface-cli login
```

### 3. Run training
```bash
python train.py
```

## ⚙️ Configuration

| Parameter | Default | Description |
|-----------|---------|-------------|
| MODEL_NAME | Qwen/Qwen2.5-0.5B-Instruct | Base model (fully open!) |
| num_train_epochs | 3 | Training iterations |
| per_device_train_batch_size | 4 | Batch size |
| gradient_accumulation_steps | 4 | Effective batch = 16 |
| learning_rate | 2e-5 | Learning rate |

## 📊 Features

- ✅ Fully open model - လိုင်စင်မလိုပါသည်။
- ✅ FP16 precision - ပိုမိုမြန်ပါသည်။
- ✅ Gradient checkpointing - Memory ချွေတာပါသည်။
- ✅ Test/Validation evaluation - နှစ်ခုလုံးအတွက် စမ်းသပ်ပါသည်။

## 📊 Training Data

Dataset: [amkyawdev/AmkyawDev-Dataset](https://huggingface.co/datasets/amkyawdev/AmkyawDev-Dataset)

| Split | Samples |
|-------|---------|
| Train | ~29,100 |
| Validation | ~29,100 |
| Test | ~29,100 |

> **Note:** Each file (train.jsonl, test.jsonl, validation.jsonl) has ~29,100 conversations!

## 💾 Output

Trained model saved to `./myanmar-qwen-output/`

## 📤 Upload to HuggingFace

```bash
cd myanmar-qwen-output
huggingface-cli upload amkyawdev/my-myanmar-qwen . --repo-type model
```

## 🖥️ Google Colab

```python
# Install
!pip install transformers datasets torch accelerate

# Login
from huggingface_hub import login
login("YOUR_TOKEN")

# Run
%run train.py
```

---
Built by amkyawdev