kamranibiyev/azNER-mBERT

azwikiner-mbert

Named entity recognition for Azerbaijani: multilingual BERT (cased) fine-tuned on azWikiNER, a manually corrected corpus of 16,740 Azerbaijani Wikipedia segments with 16 entity categories. Trained in April 2021; construction and evaluation are described in Ibiyev & Novák, TSD 2021 (doi:10.1007/978-3-030-83527-9_26) and in the author's PhD dissertation (PPKE ITK, 2026).

Entity categories

CARDINAL, EVENT, FAC, GPE, LOC, MONEY, NORP, ORDINAL, ORG, PER, PERCENT, PROD, QUANTITY, TIME, WORK_OF_ART, and TITLE (professions, roles and military ranks attached to person names, e.g. aktyor 'actor', şair 'poet'). TIME covers both times and dates; PROD corresponds to OntoNotes PRODUCT. The classifier head has two vestigial classes that do not occur in practice: an interim OTHER label (one training span) and an empty label inherited from a trailing newline in the label file.

Results

Entity-level scores on the azWikiNER test set (1,618 segments), computed with the standard conlleval script:

Tagset	Precision	Recall	F1
Full (16 types)	80.48	83.09	81.77
Common (15 types shared with zero-shot OntoNotes models)	81.70	84.10	82.89

An earlier training run preserved alongside the published one reached F1 81.56, which gives a direct estimate of run-to-run variance for this setup. Dev-set F1 (seqeval): 82.04.

Per-type F1 on the test set (full tagset):

Type	Gold spans	F1
GPE	1354	91.67
PER	519	85.74
LOC	254	68.14
ORG	248	70.14
TITLE	210	66.51
CARDINAL	146	79.35
TIME	131	81.04
WORK_OF_ART	55	52.00
FAC	52	49.06
NORP	46	56.25
EVENT	37	72.22
ORDINAL	34	81.16
QUANTITY	29	80.00
PERCENT	10	90.00
MONEY	8	31.58
PROD	8	26.67

Training

Fine-tuned with the token-classification example of transformers 4.5.0 on 4 GPUs: AdamW, learning rate 1e-4 with linear decay, no warmup, no weight decay, per-device batch size 8 (effective 32), mixed precision (fp16), seed 42, 4 epochs. Splits: 13,483 train / 1,639 dev / 1,618 test segments. The full training configuration is included in this repository as training_args.bin.

Usage

from transformers import pipeline
 
ner = pipeline("ner", model="kamranibiyev/azwikiner-mbert", aggregation_strategy="simple")
ner("Elnur Hüseynov Bakıda anadan olub.")

Limitations

The training text comes from Wikipedia; expect lower accuracy on other genres. The numeric and date-like categories entered the corpus through a zero-shot OntoNotes pass with rule-based merging rather than span-by-span human verification, so scores on those types should be read with care. Long inputs are truncated at the tokenizer limit. There is no DATE tag; dates fall under TIME.

Citation

@inproceedings{ibiyev2021azwikiner,
  author    = {Kamran Ibiyev and Attila Nov{\'a}k},
  title     = {Using Zero-Shot Transfer to Initialize azWikiNER, a Gold
               Standard Named Entity Corpus for the Azerbaijani Language},
  booktitle = {Text, Speech, and Dialogue (TSD 2021)},
  pages     = {305--317},
  publisher = {Springer},
  year      = {2021},
  doi       = {10.1007/978-3-030-83527-9_26},
}

Dataset: https://doi.org/10.5281/zenodo.22019336