Hugging Face's logo

Hailay
/
VEXMLM-Tigrinya-NER

Token Classification
Transformers
Safetensors
Tigrinya
ner
xlm-roberta
vexmlm
geez
low-resource
Model card Files
xet
 Community
VEXMLM-Tigrinya-NER
File size: 4,880 Bytes
ea3e4e3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
---
language:
- ti
license: apache-2.0
library_name: transformers
pipeline_tag: token-classification
tags:
- ner
- token-classification
- xlm-roberta
- vexmlm
- geez
- low-resource
base_model: Hailay/VEXMLM
---

# VEXMLM — Tigrinya NER

Tigrinya **token classification** fine-tuned from
[`Hailay/VEXMLM`](https://huggingface.co/Hailay/VEXMLM), the vocabulary-extended
XLM-R for Ge'ez-script languages.

Official implementation: **https://github.com/hailaykidu/VEXMLM**

| | |
|---|---|
| Task | token-classification |
| Dataset | Tigrinya NER |
| Language | Tigrinya |
| Architecture | `XLMRobertaForTokenClassification` |
| Base model | `Hailay/VEXMLM` |
| Vocabulary | 280,002 |
| Labels | 11 |
| Seeds published | 42, 43, 44, 45, 46 |

Labels cover PER, ORG, LOC, DATE and MISC in BIO format (11 classes).

## Five-seed benchmark evaluation

Fine-tuned independently under seeds 42–46 with one configuration (hash
`ce27cc194946`) on an A100-PCIE-40GB. Reported as mean ± standard deviation over
the five runs, on the dataset's **test** split.

| Metric | Score |
|---|---|
| Entity-F1 | **72.82 ± 0.79** |
| Macro-F1 | 82.19 ± 0.69 |
| Accuracy | 95.15 ± 0.05 |

These are the paper's verified results. They come from the five-seed evaluation
described above — **not** from interactive use.

### Interactive inference vs. benchmark

**Benchmark evaluation** is the five-seed measurement on the held-out test split,
shown in the table above.

**Interactive inference** is what the usage example below performs: Enter arbitrary Tigrinya text and inspect the predicted entity spans.
Predictions on arbitrary user input are demonstrations only and do not produce or
reproduce the benchmark score.

## Repository layout

Five independently fine-tuned checkpoints, one per seed. The reported benchmark
score is the mean ± standard deviation over all five; **no single seed is the
"five-seed model."**

```
seed-42/  seed-43/  seed-44/  seed-45/  seed-46/
```

Load a specific seed with the `subfolder` argument, as in the example below.

## Fine-tuning

Fine-tuned from [`Hailay/VEXMLM`](https://huggingface.co/Hailay/VEXMLM), a
vocabulary-extended XLM-R (280,002 subwords, 30,000 Ge'ez tokens merged into the
SentencePiece model) after continued MLM pretraining.

| Hyperparameter | Value |
|---|---|
| Max sequence length | 256 |
| Batch size | 32 |
| Epochs | 4 |
| Learning rate | 2e-5 |
| LR schedule | Linear decay, 10% warmup |
| Weight decay | 0.01 |
| Gradient clipping | 1.0 |
| Optimizer | AdamW (β₁ 0.9, β₂ 0.999, ε 1e-8) |
| Precision | bf16 |
| Trainable parameters | All |
| Hardware | 1× NVIDIA A100 |

Runs are bit-reproducible: `enable_full_determinism`,
`CUBLAS_WORKSPACE_CONFIG=:4096:8`, `dataloader_num_workers=0`.

## Usage

```python
from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

repo = "Hailay/VEXMLM-Tigrinya-NER"
tokenizer = AutoTokenizer.from_pretrained(repo, subfolder="seed-42")
model = AutoModelForTokenClassification.from_pretrained(repo, subfolder="seed-42")
model.eval()

words = "ኤርትራ ኣብ ቀርኒ አፍሪቃ እትርከብ ሃገር እያ።".split()
enc = tokenizer(words, is_split_into_words=True, return_tensors="pt", truncation=True)

with torch.no_grad():
    pred = model(**enc).logits.argmax(-1)[0].tolist()

seen = set()
for p, w in zip(pred, enc.word_ids(0)):
    if w is None or w in seen:
        continue
    seen.add(w)
    print(words[w], "->", model.config.id2label[p])
```

## Limitations

- Fine-tuned for Tigrinya on Tigrinya NER only; performance on other
  languages, domains or label schemes is not characterised.
- The base model covers Amharic and Tigrinya; other Ge'ez-script languages were
  not part of pretraining.
- Corpora are drawn largely from religious and news domains, and the model may
  reflect those distributions and any biases in them.
- Single-configuration study: no hyperparameter search was performed, and
  baseline comparisons in the paper are single-seed.

## Reproducibility

The fine-tuning launcher, evaluation code and per-run result records are in the
official repository: **https://github.com/hailaykidu/VEXMLM**

```bash
sbatch scripts/slurm_stage2_spm_seeds.sh    # 6 tasks × 5 seeds
python3 evaluation/export_spm_results.py    # regenerates the metrics table
```

## Citation

```bibtex
@inproceedings{teklehaymanot2026vexmlm,
  title     = {Expanding the Lexicon of Ge'ez Based African Languages:
               A Comparative Study of Amharic and Tigrinya},
  author    = {Teklehaymanot, Hailay Kidu and Yadeta, Gebregziabihier and
               Nejdl, Wolfgang},
  booktitle = {Proceedings of the Workshop on Language Models for
               Underserved Communities (LM4UC) at IJCAI},
  year      = {2026}
}
```

Accepted at the LM4UC Workshop, IJCAI 2026.

## License

Apache 2.0, following `xlm-roberta-base`.