Update README.md

be49435 verified 5 days ago

10 kB

---
license: other
base_model: google/gemma-3-1b-it
library_name: peft
pipeline_tag: text-generation
tags:
  - balochi
  - english-to-balochi
  - translation
  - machine-translation
  - gemma
  - lora
  - peft
  - low-resource-language
---
# BalochiBridge 🌉
### English-to-Balochi Machine Translation Using Gemma 3 1B and LoRA

BalochiBridge is an AI-powered English-to-Balochi translation model developed using Google's Gemma 3 1B Instruct model and Low-Rank Adaptation (LoRA).

The project adapts a pretrained language model for English-to-Balochi translation using a Balochi language dataset. It aims to support Natural Language Processing (NLP) research and contribute to the development of AI tools for the Balochi language.

**This repository contains the fine-tuned LoRA adapter. It does not contain the complete Gemma base model.**

## 1. Model Details

| Property | Description |
|---|---|
| Model Name | BalochiBridge |
| Task | English-to-Balochi Translation |
| Base Model | Google Gemma 3 1B Instruct |
| Base Model ID | `google/gemma-3-1b-it` |
| Fine-Tuning Method | LoRA |
| Framework | Hugging Face Transformers |
| Fine-Tuning Library | PEFT |
| Quantization During Training | 4-bit NF4 |
| Dataset | `Aynkader/Balochi-dataset` |
| Model Type | LoRA Adapter |

## 2. Important: Gemma Access Requirement

The base model used by BalochiBridge is a gated Hugging Face model.

Before running BalochiBridge, you must:

1. Create or sign in to your Hugging Face account.
2. Visit [Google Gemma 3 1B Instruct](https://huggingface.co/google/gemma-3-1b-it).
3. Accept Google's model access conditions.
4. Wait until your account has been granted access.

**Important:** You need access to the Gemma base model even though the BalochiBridge adapter repository is public.

You also need a Hugging Face access token with read permission. Your token is private and must never be shared.

## 3. Run BalochiBridge Directly in Google Colab

The easiest way to use BalochiBridge is through Google Colab.

**Recommended environment:**
- Google Colab
- Python 3
- GPU runtime (T4 or better recommended)
- Internet connection
- Hugging Face account with Gemma access

### Step 1: Enable GPU

Open [Google Colab](https://colab.research.google.com/).

Create a new notebook.

Select:

`Runtime → Change runtime type → T4 GPU`

Save the settings.

### Step 2: Install Dependencies

Run this cell:

```python
!pip uninstall -y torchao -q
!pip install -q -U \
    transformers \
    peft \
    accelerate \
    sentencepiece \
    huggingface_hub
```

**Important:** After installation, restart the Colab runtime using:

`Runtime → Restart session`

This prevents old packages from remaining loaded in memory.

### Step 3: Authenticate with Hugging Face

Run:

```python
from huggingface_hub import notebook_login, whoami
notebook_login()
account = whoami()
print("Logged in successfully!")
print("Hugging Face account:", account["name"])
```

When prompted, enter your own Hugging Face access token.

### Step 4: Load the Base Model and BalochiBridge Adapter

Run:

```python
import torch
from transformers import (
    AutoTokenizer,
    AutoModelForCausalLM
)
from peft import PeftModel
BASE_MODEL = "google/gemma-3-1b-it"
ADAPTER_MODEL = "ullahhafeez321/BalochiBridge"
# Check GPU availability
if not torch.cuda.is_available():
    raise RuntimeError(
        "GPU not detected. Enable a GPU runtime in Google Colab."
    )
print("GPU:", torch.cuda.get_device_name(0))
# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(
    BASE_MODEL
)
# Load pretrained Gemma model
base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    torch_dtype=torch.float16,
    device_map="auto"
)
# Attach BalochiBridge LoRA adapter
model = PeftModel.from_pretrained(
    base_model,
    ADAPTER_MODEL
)
model.eval()
print("BalochiBridge loaded successfully!")
```

This loads the pretrained Gemma model and attaches the BalochiBridge adapter. No training or fine-tuning is required.

### Step 5: Create the Translation Function

Run:

```python
import torch
def translate_to_balochi(
    english_text,
    max_new_tokens=128
):
    if not english_text.strip():
        return "Please enter an English sentence."
    prompt = f"""Translate the following English text into Balochi.
Only provide the Balochi translation. Do not explain.
English: {english_text}
Balochi:"""
    inputs = tokenizer(
        prompt,
        return_tensors="pt"
    )
    # Use the device where the model's input embeddings are located
    input_device = model.get_input_embeddings().weight.device
    inputs = inputs.to(input_device)
    with torch.inference_mode():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            do_sample=False,
            repetition_penalty=1.1,
            pad_token_id=tokenizer.eos_token_id,
            eos_token_id=tokenizer.eos_token_id
        )
    # Extract only the generated translation
    generated_tokens = outputs[
        0,
        inputs["input_ids"].shape[1]:
    ]
    translation = tokenizer.decode(
        generated_tokens,
        skip_special_tokens=True
    ).strip()
    return translation
```

### Step 6: Translate Your First Sentence

Run:

```python
english = "Hello, how are you?"
translation = translate_to_balochi(english)
print("English:", english)
print("Balochi:", translation)
```

### Step 7: Translate Multiple Sentences

```python
test_sentences = [
    "Hello, how are you?",
    "My name is Ahmed.",
    "I live in Gwadar.",
    "I love my country.",
    "What is your name?",
    "Thank you very much.",
    "Today the weather is beautiful.",
    "I am learning artificial intelligence."
]
for sentence in test_sentences:
    translation = translate_to_balochi(sentence)
    print("\nEnglish:", sentence)
    print("Balochi:", translation)
    print("-" * 60)
```

### Step 8: Interactive Translation

You can also enter your own English sentences:

```python
while True:
    english = input(
        "\nEnter an English sentence (or type 'exit'): "
    )
    if english.lower().strip() == "exit":
        print("Translation session ended.")
        break
    translation = translate_to_balochi(english)
    print("\nBalochi Translation:")
    print(translation)
```

## 4. How BalochiBridge Works

The model follows this workflow:

1. **Input:** An English sentence is provided.
2. **Tokenization:** The tokenizer converts the input into tokens.
3. **Base Model:** Gemma 3 1B processes the input.
4. **LoRA Adapter:** The trained BalochiBridge adapter adapts the model for Balochi translation.
5. **Generation:** The model generates the Balochi translation.
6. **Output:** The generated tokens are decoded into readable text.

## 5. Training and Fine-Tuning

BalochiBridge was developed using Parameter-Efficient Fine-Tuning (PEFT), specifically Low-Rank Adaptation (LoRA).

LoRA makes it possible to adapt a pretrained model without updating all its original parameters. Instead, it introduces smaller trainable matrices that learn task-specific information.

The base model was loaded using 4-bit NF4 quantization during the training workflow to reduce GPU memory requirements.

**The published repository contains the resulting adapter weights, allowing users to use the trained model without repeating the training process.**

## 6. Evaluation Metrics

The following metrics can be used to evaluate BalochiBridge:

| Metric | Description |
|---|---|
| BLEU | Measures n-gram overlap between generated and reference translations. |
| chrF | Measures character-level similarity. |
| ROUGE-L | Measures the longest common subsequence. |
| CER | Measures character-level errors. |
| Exact Match | Measures the percentage of predictions that exactly match the reference. |

Higher BLEU, chrF, and ROUGE-L scores generally indicate greater similarity to reference translations. Lower CER indicates fewer character-level errors.

Actual evaluation scores should be reported separately using the results obtained from the project's test dataset.

## 7. Repository Contents

The Hugging Face repository contains the LoRA adapter and associated configuration and tokenizer files.

| File | Purpose |
|---|---|
| `adapter_model.safetensors` | Trained LoRA adapter weights |
| `adapter_config.json` | LoRA adapter configuration |
| `tokenizer.json` | Tokenizer vocabulary and configuration data |
| `tokenizer_config.json` | Tokenizer settings |
| `chat_template.jinja` | Chat formatting template |
| `training_args.bin` | Saved training arguments |
| `README.md` | Model documentation |

## 8. Limitations

- Translation quality depends on the quality and diversity of the training dataset.
- Balochi contains dialect and spelling variations.
- Low-resource language translation remains challenging.
- Some generated translations may contain grammatical or semantic errors.
- Human evaluation by fluent Balochi speakers is recommended.
- The model is intended for English-to-Balochi translation, not general-purpose translation.

## 9. Future Improvements

- Expand the English-Balochi parallel dataset.
- Improve translation quality across Balochi dialects.
- Perform additional evaluation using a larger independent test dataset.
- Improve grammatical accuracy and sentence fluency.
- Develop a web-based translation application.
- Explore additional fine-tuning and optimization methods.

## 10. Contributors

- Hafsa
- Abdullah

## 11. Acknowledgements

- [Google Gemma](https://huggingface.co/google/gemma-3-1b-it) — pretrained language model.
- [Hugging Face](https://huggingface.co/) — model hosting and machine learning libraries.
- [Aynkader/Balochi-dataset](https://huggingface.co/datasets/Aynkader/Balochi-dataset) — Balochi dataset.

---

**BalochiBridge — Building a bridge between English and Balochi through Artificial Intelligence.**

Model repository: [ullahhafeez321/BalochiBridge](https://huggingface.co/ullahhafeez321/BalochiBridge)

	---
	license: other
	base_model: google/gemma-3-1b-it
	library_name: peft
	pipeline_tag: text-generation
	tags:
	- balochi
	- english-to-balochi
	- translation
	- machine-translation
	- gemma
	- lora
	- peft
	- low-resource-language
	---

	# BalochiBridge 🌉
	### English-to-Balochi Machine Translation Using Gemma 3 1B and LoRA

	BalochiBridge is an AI-powered English-to-Balochi translation model developed using Google's Gemma 3 1B Instruct model and Low-Rank Adaptation (LoRA).

	The project adapts a pretrained language model for English-to-Balochi translation using a Balochi language dataset. It aims to support Natural Language Processing (NLP) research and contribute to the development of AI tools for the Balochi language.

	This repository contains the fine-tuned LoRA adapter. It does not contain the complete Gemma base model.

	## 1. Model Details

	\| Property \| Description \|
	\|---\|---\|
	\| Model Name \| BalochiBridge \|
	\| Task \| English-to-Balochi Translation \|
	\| Base Model \| Google Gemma 3 1B Instruct \|
	\| Base Model ID \| `google/gemma-3-1b-it` \|
	\| Fine-Tuning Method \| LoRA \|
	\| Framework \| Hugging Face Transformers \|
	\| Fine-Tuning Library \| PEFT \|
	\| Quantization During Training \| 4-bit NF4 \|
	\| Dataset \| `Aynkader/Balochi-dataset` \|
	\| Model Type \| LoRA Adapter \|

	## 2. Important: Gemma Access Requirement

	The base model used by BalochiBridge is a gated Hugging Face model.

	Before running BalochiBridge, you must:

	1. Create or sign in to your Hugging Face account.
	2. Visit [Google Gemma 3 1B Instruct](https://huggingface.co/google/gemma-3-1b-it).
	3. Accept Google's model access conditions.
	4. Wait until your account has been granted access.

	Important: You need access to the Gemma base model even though the BalochiBridge adapter repository is public.

	You also need a Hugging Face access token with read permission. Your token is private and must never be shared.

	## 3. Run BalochiBridge Directly in Google Colab

	The easiest way to use BalochiBridge is through Google Colab.

	Recommended environment:
	- Google Colab
	- Python 3
	- GPU runtime (T4 or better recommended)
	- Internet connection
	- Hugging Face account with Gemma access

	### Step 1: Enable GPU

	Open [Google Colab](https://colab.research.google.com/).

	Create a new notebook.

	Select:

	`Runtime → Change runtime type → T4 GPU`

	Save the settings.

	### Step 2: Install Dependencies

	Run this cell:

	```python
	!pip uninstall -y torchao -q

	!pip install -q -U \
	transformers \
	peft \
	accelerate \
	sentencepiece \
	huggingface_hub
	```

	Important: After installation, restart the Colab runtime using:

	`Runtime → Restart session`

	This prevents old packages from remaining loaded in memory.

	### Step 3: Authenticate with Hugging Face

	Run:

	```python
	from huggingface_hub import notebook_login, whoami

	notebook_login()

	account = whoami()

	print("Logged in successfully!")
	print("Hugging Face account:", account["name"])
	```

	When prompted, enter your own Hugging Face access token.

	### Step 4: Load the Base Model and BalochiBridge Adapter

	Run:

	```python
	import torch

	from transformers import (
	AutoTokenizer,
	AutoModelForCausalLM
	)

	from peft import PeftModel

	BASE_MODEL = "google/gemma-3-1b-it"
	ADAPTER_MODEL = "ullahhafeez321/BalochiBridge"

	# Check GPU availability
	if not torch.cuda.is_available():
	raise RuntimeError(
	"GPU not detected. Enable a GPU runtime in Google Colab."
	)

	print("GPU:", torch.cuda.get_device_name(0))

	# Load tokenizer
	tokenizer = AutoTokenizer.from_pretrained(
	BASE_MODEL
	)

	# Load pretrained Gemma model
	base_model = AutoModelForCausalLM.from_pretrained(
	BASE_MODEL,
	torch_dtype=torch.float16,
	device_map="auto"
	)

	# Attach BalochiBridge LoRA adapter
	model = PeftModel.from_pretrained(
	base_model,
	ADAPTER_MODEL
	)

	model.eval()

	print("BalochiBridge loaded successfully!")
	```

	This loads the pretrained Gemma model and attaches the BalochiBridge adapter. No training or fine-tuning is required.

	### Step 5: Create the Translation Function

	Run:

	```python
	import torch

	def translate_to_balochi(
	english_text,
	max_new_tokens=128
	):
	if not english_text.strip():
	return "Please enter an English sentence."

	prompt = f"""Translate the following English text into Balochi.
	Only provide the Balochi translation. Do not explain.

	English: {english_text}
	Balochi:"""

	inputs = tokenizer(
	prompt,
	return_tensors="pt"
	)

	# Use the device where the model's input embeddings are located
	input_device = model.get_input_embeddings().weight.device
	inputs = inputs.to(input_device)

	with torch.inference_mode():
	outputs = model.generate(
	**inputs,
	max_new_tokens=max_new_tokens,
	do_sample=False,
	repetition_penalty=1.1,
	pad_token_id=tokenizer.eos_token_id,
	eos_token_id=tokenizer.eos_token_id
	)

	# Extract only the generated translation
	generated_tokens = outputs[
	0,
	inputs["input_ids"].shape[1]:
	]

	translation = tokenizer.decode(
	generated_tokens,
	skip_special_tokens=True
	).strip()

	return translation
	```

	### Step 6: Translate Your First Sentence

	Run:

	```python
	english = "Hello, how are you?"

	translation = translate_to_balochi(english)

	print("English:", english)
	print("Balochi:", translation)
	```

	### Step 7: Translate Multiple Sentences

	```python
	test_sentences = [
	"Hello, how are you?",
	"My name is Ahmed.",
	"I live in Gwadar.",
	"I love my country.",
	"What is your name?",
	"Thank you very much.",
	"Today the weather is beautiful.",
	"I am learning artificial intelligence."
	]

	for sentence in test_sentences:

	translation = translate_to_balochi(sentence)

	print("\nEnglish:", sentence)
	print("Balochi:", translation)
	print("-" * 60)
	```

	### Step 8: Interactive Translation

	You can also enter your own English sentences:

	```python
	while True:

	english = input(
	"\nEnter an English sentence (or type 'exit'): "
	)

	if english.lower().strip() == "exit":
	print("Translation session ended.")
	break

	translation = translate_to_balochi(english)

	print("\nBalochi Translation:")
	print(translation)
	```

	## 4. How BalochiBridge Works

	The model follows this workflow:

	1. Input: An English sentence is provided.
	2. Tokenization: The tokenizer converts the input into tokens.
	3. Base Model: Gemma 3 1B processes the input.
	4. LoRA Adapter: The trained BalochiBridge adapter adapts the model for Balochi translation.
	5. Generation: The model generates the Balochi translation.
	6. Output: The generated tokens are decoded into readable text.

	## 5. Training and Fine-Tuning

	BalochiBridge was developed using Parameter-Efficient Fine-Tuning (PEFT), specifically Low-Rank Adaptation (LoRA).

	LoRA makes it possible to adapt a pretrained model without updating all its original parameters. Instead, it introduces smaller trainable matrices that learn task-specific information.

	The base model was loaded using 4-bit NF4 quantization during the training workflow to reduce GPU memory requirements.

	The published repository contains the resulting adapter weights, allowing users to use the trained model without repeating the training process.

	## 6. Evaluation Metrics

	The following metrics can be used to evaluate BalochiBridge:

	\| Metric \| Description \|
	\|---\|---\|
	\| BLEU \| Measures n-gram overlap between generated and reference translations. \|
	\| chrF \| Measures character-level similarity. \|
	\| ROUGE-L \| Measures the longest common subsequence. \|
	\| CER \| Measures character-level errors. \|
	\| Exact Match \| Measures the percentage of predictions that exactly match the reference. \|

	Higher BLEU, chrF, and ROUGE-L scores generally indicate greater similarity to reference translations. Lower CER indicates fewer character-level errors.

	Actual evaluation scores should be reported separately using the results obtained from the project's test dataset.

	## 7. Repository Contents

	The Hugging Face repository contains the LoRA adapter and associated configuration and tokenizer files.

	\| File \| Purpose \|
	\|---\|---\|
	\| `adapter_model.safetensors` \| Trained LoRA adapter weights \|
	\| `adapter_config.json` \| LoRA adapter configuration \|
	\| `tokenizer.json` \| Tokenizer vocabulary and configuration data \|
	\| `tokenizer_config.json` \| Tokenizer settings \|
	\| `chat_template.jinja` \| Chat formatting template \|
	\| `training_args.bin` \| Saved training arguments \|
	\| `README.md` \| Model documentation \|

	## 8. Limitations

	- Translation quality depends on the quality and diversity of the training dataset.
	- Balochi contains dialect and spelling variations.
	- Low-resource language translation remains challenging.
	- Some generated translations may contain grammatical or semantic errors.
	- Human evaluation by fluent Balochi speakers is recommended.
	- The model is intended for English-to-Balochi translation, not general-purpose translation.

	## 9. Future Improvements

	- Expand the English-Balochi parallel dataset.
	- Improve translation quality across Balochi dialects.
	- Perform additional evaluation using a larger independent test dataset.
	- Improve grammatical accuracy and sentence fluency.
	- Develop a web-based translation application.
	- Explore additional fine-tuning and optimization methods.

	## 10. Contributors

	- Hafsa
	- Abdullah

	## 11. Acknowledgements

	- [Google Gemma](https://huggingface.co/google/gemma-3-1b-it) — pretrained language model.
	- [Hugging Face](https://huggingface.co/) — model hosting and machine learning libraries.
	- [Aynkader/Balochi-dataset](https://huggingface.co/datasets/Aynkader/Balochi-dataset) — Balochi dataset.

	---

	BalochiBridge — Building a bridge between English and Balochi through Artificial Intelligence.

	Model repository: [ullahhafeez321/BalochiBridge](https://huggingface.co/ullahhafeez321/BalochiBridge)