Hasanblanka/polygraf-ner-stage4-final-model

Polygraf NER Stage 4 Final Model

This is the final improved English NER model produced for Stage 4 of the Polygraf Applied NLP / NER technical task. It fine-tunes distilbert/distilbert-base-cased on the reviewed and expanded Stage 4 dataset.

Labels

The model predicts BIO tags for eight flat, non-overlapping entity types:

• PERSON
• ORGANIZATION
• LOCATION
• TIMEDATE
• PRODUCT
• WORKOFART
• JOB
• AMOUNT

The exact id2label and label2id mappings are stored in config.json.

Architecture

• Architecture: DistilBertForTokenClassification
• Parameters: 65,203,985
• Hidden size: 768
• Transformer layers: 6
• Attention heads: 12
• Activation: GELU, inherited from DistilBERT
• Saved weights: approximately 260.8 MB
• Base revision: 6ea81172465e8b0ad3fddeed32b986cdcdcffcf0

Training data

• Dataset: Hasanblanka/polygraf-ner-stage4-final-dataset
• Train records: 1,954
• Test records: 472
• Validation records: 0
• Earlier train records moved to test: 0
• Exact train/test text leakage: 0
• Global Voices article-group leakage: 0

The test split combines 127 frozen earlier test records with 345 newly held-out Global Voices records. The newly held-out records were selected only from previously unseen article groups.

Training configuration

• Epochs: 4
• Batch size: 8
• Gradient accumulation: 2
• Effective batch size: 16
• Optimizer steps: 492
• Optimizer: AdamW
• Learning rate: 5e-5
• Weight decay: 0.01
• Warmup ratio: 0.1
• Maximum gradient norm: 1.0
• Scheduler: linear schedule with warmup
• Seed: 42
• Maximum sequence length: 512
• Truncation: disabled
• Device: CPU
• Training runtime: approximately 1 hour 45 minutes

No validation split was used. The held-out test split did not contribute gradients.

Evaluation

Metric	Train	Combined test
Precision	0.9041	0.7077
Recall	0.9511	0.8100
Entity F1	0.9270	0.7554
Macro F1	0.9019	0.7022
Token accuracy	0.9919	0.9571

Test results by entity type:

Label	Precision	Recall	F1
PERSON	0.5815	0.7133	0.6407
ORGANIZATION	0.5951	0.7295	0.6555
LOCATION	0.7308	0.8732	0.7957
TIMEDATE	0.8549	0.9083	0.8808
PRODUCT	0.6752	0.6529	0.6639
WORKOFART	0.2848	0.3849	0.3274
JOB	0.7057	0.8098	0.7542
AMOUNT	0.8900	0.9100	0.8999

For an apples-to-apples comparison, entity F1 on the same 127 frozen records rose from 0.5530 in Stage 4 Part 1 to 0.5967 in this final model. The newly held-out 345-record Global Voices subset scored 0.7929.

Frozen wild-sample QA

The same 10 manually annotated samples used to diagnose the Stage 3 model were evaluated again:

• Precision: 0.7872
• Recall: 0.8605
• Entity F1: 0.8222
• Macro F1: 0.7794

Stage 3 scored 0.5376 F1 on this set. The final model improved organization, job, temporal, and excessive work-title behavior, while product boundaries and some organization/location distinctions remain unresolved.

Usage

from transformers import pipeline

model_id = "Hasanblanka/polygraf-ner-stage4-final-model"
ner = pipeline(
    task="token-classification",
    model=model_id,
    tokenizer=model_id,
    aggregation_strategy="simple",
)

text = "Maya Ortiz joined Northstar Robotics in Lisbon on March 12, 2026."
for entity in ner(text):
    print(entity)

Reproducibility files

• stage4_part2_model.yaml: pinned training configuration.
• metrics/train_metrics.json: complete train metrics.
• metrics/test_metrics.json: complete combined-test metrics.
• metrics/per_label_*.json: label-level metrics.
• metrics/manual_test_metrics.json: frozen wild-test metrics.
• metrics/training_history.json: per-epoch loss and learning-rate history.
• metrics/training_summary.json: environment, hashes, runtime, and outputs.

Limitations

• This model is not intended for production use.
• WORKOFART remains the weakest label and shows boundary fragmentation, false positives, and missed titles.
• PERSON and ORGANIZATION generalization is weaker than the temporal and amount labels.
• Train F1 is substantially higher than test F1, indicating a remaining generalization gap.
• Training data is English and news-heavy.
• Nested entities are not supported.
• The 10-record wild test is diagnostic, not statistically representative.

License and data terms

The base DistilBERT checkpoint is Apache-2.0. The fine-tuning dataset combines evaluation-use-only starter data, CC BY / CC BY-SA Open Newswire material, CC BY 3.0 Global Voices passages, and synthetic records. This model is published with license: other for evaluation and research. Users must review the full licensing and attribution notes in the Stage 4 dataset repository.