README.md

---
language:
- ti
license: apache-2.0
library_name: transformers
pipeline_tag: token-classification
tags:
- ner
- token-classification
- xlm-roberta
- vexmlm
- geez
- low-resource
base_model: Hailay/VEXMLM
---

# VEXMLM — Tigrinya NER

Tigrinya **token classification** fine-tuned from
[`Hailay/VEXMLM`](https://huggingface.co/Hailay/VEXMLM), the vocabulary-extended
XLM-R for Ge'ez-script languages.

Official implementation: **https://github.com/hailaykidu/VEXMLM**

| | |
|---|---|
| Task | token-classification |
| Dataset | Tigrinya NER |
| Language | Tigrinya |
| Architecture | `XLMRobertaForTokenClassification` |
| Base model | `Hailay/VEXMLM` |
| Vocabulary | 280,002 |
| Labels | 11 |
| Seeds published | 42, 43, 44, 45, 46 |

Labels cover PER, ORG, LOC, DATE and MISC in BIO format (11 classes).

## Five-seed benchmark evaluation

Fine-tuned independently under seeds 42–46 with one configuration (hash
`ce27cc194946`) on an A100-PCIE-40GB. Reported as mean ± standard deviation over
the five runs, on the dataset's **test** split.

| Metric | Score |
|---|---|
| Entity-F1 | **72.82 ± 0.79** |
| Macro-F1 | 82.19 ± 0.69 |
| Accuracy | 95.15 ± 0.05 |

These are the paper's verified results. They come from the five-seed evaluation
described above — **not** from interactive use.

### Interactive inference vs. benchmark

**Benchmark evaluation** is the five-seed measurement on the held-out test split,
shown in the table above.

**Interactive inference** is what the usage example below performs: Enter arbitrary Tigrinya text and inspect the predicted entity spans.
Predictions on arbitrary user input are demonstrations only and do not produce or
reproduce the benchmark score.

## Repository layout

Five independently fine-tuned checkpoints, one per seed. The reported benchmark
score is the mean ± standard deviation over all five; **no single seed is the
"five-seed model."**

```
seed-42/  seed-43/  seed-44/  seed-45/  seed-46/
```

Load a specific seed with the `subfolder` argument, as in the example below.

## Fine-tuning

Fine-tuned from [`Hailay/VEXMLM`](https://huggingface.co/Hailay/VEXMLM), a
vocabulary-extended XLM-R (280,002 subwords, 30,000 Ge'ez tokens merged into the
SentencePiece model) after continued MLM pretraining.

| Hyperparameter | Value |
|---|---|
| Max sequence length | 256 |
| Batch size | 32 |
| Epochs | 4 |
| Learning rate | 2e-5 |
| LR schedule | Linear decay, 10% warmup |
| Weight decay | 0.01 |
| Gradient clipping | 1.0 |
| Optimizer | AdamW (β₁ 0.9, β₂ 0.999, ε 1e-8) |
| Precision | bf16 |
| Trainable parameters | All |
| Hardware | 1× NVIDIA A100 |

Runs are bit-reproducible: `enable_full_determinism`,
`CUBLAS_WORKSPACE_CONFIG=:4096:8`, `dataloader_num_workers=0`.

## Usage

```python
from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

repo = "Hailay/VEXMLM-Tigrinya-NER"
tokenizer = AutoTokenizer.from_pretrained(repo, subfolder="seed-42")
model = AutoModelForTokenClassification.from_pretrained(repo, subfolder="seed-42")
model.eval()

words = "ኤርትራ ኣብ ቀርኒ አፍሪቃ እትርከብ ሃገር እያ።".split()
enc = tokenizer(words, is_split_into_words=True, return_tensors="pt", truncation=True)

with torch.no_grad():
    pred = model(**enc).logits.argmax(-1)[0].tolist()

seen = set()
for p, w in zip(pred, enc.word_ids(0)):
    if w is None or w in seen:
        continue
    seen.add(w)
    print(words[w], "->", model.config.id2label[p])
```

## Limitations

- Fine-tuned for Tigrinya on Tigrinya NER only; performance on other
  languages, domains or label schemes is not characterised.
- The base model covers Amharic and Tigrinya; other Ge'ez-script languages were
  not part of pretraining.
- Corpora are drawn largely from religious and news domains, and the model may
  reflect those distributions and any biases in them.
- Single-configuration study: no hyperparameter search was performed, and
  baseline comparisons in the paper are single-seed.

## Reproducibility

The fine-tuning launcher, evaluation code and per-run result records are in the
official repository: **https://github.com/hailaykidu/VEXMLM**

```bash
sbatch scripts/slurm_stage2_spm_seeds.sh    # 6 tasks × 5 seeds
python3 evaluation/export_spm_results.py    # regenerates the metrics table
```

## Citation

```bibtex
@inproceedings{teklehaymanot2026vexmlm,
  title     = {Expanding the Lexicon of Ge'ez Based African Languages:
               A Comparative Study of Amharic and Tigrinya},
  author    = {Teklehaymanot, Hailay Kidu and Yadeta, Gebregziabihier and
               Nejdl, Wolfgang},
  booktitle = {Proceedings of the Workshop on Language Models for
               Underserved Communities (LM4UC) at IJCAI},
  year      = {2026}
}
```

Accepted at the LM4UC Workshop, IJCAI 2026.

## License

Apache 2.0, following `xlm-roberta-base`.