# Bilingual Translation Evaluation Script (EN → KK)

This repository provides an evaluation pipeline for English-to-Kazakh/Russian-to-Kazakh (and vice versa) translation models based on the `Gemma3ForCausalLM` architecture from Hugging Face Transformers.

## 🚀 Overview

The script:
- Loads a fine-tuned model and tokenizer
- Performs inference on a FLORES-style test set (`.jsonl`)
- Computes BLEU score using NLTK
- Saves predictions and evaluation results into a JSON file

## ⚙️ Configuration

Modify these lines at the top of the script as needed:

```python
SRC_LANG = "en"
TGT_LANG = "kk"
MODEL_PATH = "/path/to/your/model"
TEST_FILE = "/path/to/test_file.jsonl"
OUTPUT_JSON = "/path/to/output_file.jsonl"
MAX_NEW_TOKS = 64
DEVICE = "cuda"  # or "cpu"
```

To specify GPU devices:
```bash
export CUDA_VISIBLE_DEVICES=2,3,4,5
```

## ▶️ Run the Script

```bash
python eval_blue.py
```