SFT-Data for Walia-LLM: Enhancing Amharic-LLaMA by Integrating Task-Specific and Generative Datasets

Dataset Summary

The Walia dataset is designed to enhance large language models for the Amharic language by:

• Converting existing task-specific datasets (e.g., sentiment analysis, QA, NER) into instruction format.
• Creating new generative datasets (e.g., poem generation, religious lyrics, story generation).
• Translating English instruction datasets (e.g., Alpaca, Dolly) into Amharic for comparative studies.

Each data point follows a structured instruction format with:

• "instruction" – a natural language task description,
• "input" – optional input text for the task,
• "output" – the expected model output in Amharic.

Supported Tasks

Dataset Structure

{
  "instruction": "Translate the following sentence to Amharic.",
  "input": "Hello, how are you?",
  "output": "ሰላም፣ እንዴት ነህ?"
}

Data Statistics

• ~122,000 instruction samples for training
• ~15,000 for validation and test
• 16+ task types and instruction templates
• All responses are in Amharic (except source text in MT)

How to Use

You can load the dataset using the Hugging Face datasets library:

from datasets import load_dataset

dataset = load_dataset("EthioNLP/walia-amharic-instructions")
print(dataset["train"][0])

Applications

• Supervised fine-tuning (SFT) of LLMs for Amharic
• Cross-lingual instruction tuning experiments
• Evaluation of generative capabilities in low-resource languages

Related Models

The dataset is used to fine-tune:

• EthioNLP/Amharic-LLAMA-all-data
• EthioNLP/Amharic_LLAMA_our_data

Citation

Please cite the following paper if you use this dataset:

@inproceedings{azime-etal-2024-walia,
    title = "Walia-{LLM}: Enhancing {A}mharic-{LL}a{MA} by Integrating Task-Specific and Generative Datasets",
    author = "Azime, Israel Abebe  and Tonja, Atnafu Lambebo  and Belay, Tadesse Destaw  and Fuge, Mitiku Yohannes  and Wassie, Aman Kassahun  and Jada, Eyasu Shiferaw  and Chanie, Yonas  and Sewunetie, Walelign Tewabe  and Yimam, Seid Muhie",
    editor = "Al-Onaizan, Yaser  and Bansal, Mohit  and Chen, Yun-Nung",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2024",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.findings-emnlp.25/",
    doi = "10.18653/v1/2024.findings-emnlp.25",
    pages = "432--444"
}