kamranibiyev/azNER-xlmr

azwikiner-xlmr

Named entity recognition for Azerbaijani: XLM-RoBERTa base fine-tuned on azWikiNER, a manually corrected corpus of 16,740 Azerbaijani Wikipedia segments with 16 entity categories. Trained in April 2021; construction and evaluation are described in Ibiyev & Novák, TSD 2021 (doi:10.1007/978-3-030-83527-9_26) and in the author's PhD dissertation (PPKE ITK, 2026).

Entity categories

CARDINAL, EVENT, FAC, GPE, LOC, MONEY, NORP, ORDINAL, ORG, PER, PERCENT, PROD, QUANTITY, TIME, WORK_OF_ART, and TITLE (professions, roles and military ranks attached to person names, e.g. aktyor 'actor', şair 'poet'). TIME covers both times and dates; PROD corresponds to OntoNotes PRODUCT. The classifier head has two vestigial classes that do not occur in practice: an interim OTHER label (one training span) and an empty label inherited from a trailing newline in the label file.

Results

Entity-level scores on the azWikiNER test set (1,618 segments), computed with the standard conlleval script:

Tagset	Precision	Recall	F1
Full (16 types)	80.21	83.38	81.77
Common (15 types shared with zero-shot OntoNotes models)	81.30	84.37	82.81

Per-type F1 on the test set (full tagset):

Type	Gold spans	F1
GPE	1354	92.05
PER	519	86.45
LOC	254	66.93
ORG	248	68.95
TITLE	210	67.44
CARDINAL	146	77.74
TIME	131	77.61
WORK_OF_ART	55	43.64
FAC	52	57.41
NORP	46	55.77
EVENT	37	81.16
ORDINAL	34	81.69
QUANTITY	29	81.36
PERCENT	10	100.00
MONEY	8	40.00
PROD	8	16.67

Training

Fine-tuned with the token-classification example of transformers 4.5.0 on 4 GPUs: AdamW, learning rate 1e-4 with linear decay, no warmup, no weight decay, per-device batch size 8 (effective 32), mixed precision (fp16), seed 42, 6 epochs. Splits: 13,483 train / 1,639 dev / 1,618 test segments. The full training configuration is included in this repository as training_args.bin.

Usage

from transformers import pipeline
 
ner = pipeline("ner", model="kamranibiyev/azwikiner-xlmr", aggregation_strategy="simple")
ner("Elnur Hüseynov Bakıda anadan olub.")

Limitations

The training text comes from Wikipedia; expect lower accuracy on other genres. The numeric and date-like categories entered the corpus through a zero-shot OntoNotes pass with rule-based merging rather than span-by-span human verification, so scores on those types should be read with care. Long inputs are truncated at the tokenizer limit. There is no DATE tag; dates fall under TIME.

Citation

@inproceedings{ibiyev2021azwikiner,
  author    = {Kamran Ibiyev and Attila Nov{\'a}k},
  title     = {Using Zero-Shot Transfer to Initialize azWikiNER, a Gold
               Standard Named Entity Corpus for the Azerbaijani Language},
  booktitle = {Text, Speech, and Dialogue (TSD 2021)},
  pages     = {305--317},
  publisher = {Springer},
  year      = {2021},
  doi       = {10.1007/978-3-030-83527-9_26},
}

Dataset: https://doi.org/10.5281/zenodo.22019336