README.md

---
language:
- en
library_name: transformers
pipeline_tag: token-classification
tags:
- named-entity-recognition
- ner
- token-classification
- distilbert
- bio
base_model: distilbert/distilbert-base-cased
datasets:
- Hasanblanka/polygraf-ner-stage_2-dataset
license: other
model-index:
- name: Polygraf NER Stage 3 First Model
  results:
  - task:
      type: token-classification
      name: Named Entity Recognition
    dataset:
      name: Polygraf NER Stage 2 Expanded
      type: Hasanblanka/polygraf-ner-stage_2-dataset
      split: test
    metrics:
    - type: precision
      value: 0.547275
    - type: recall
      value: 0.625954
    - type: f1
      value: 0.583976
---

# Polygraf NER Stage 3 First Model

This is the first English NER model produced for Stage 3 of the Polygraf
Applied NLP / NER technical task. It fine-tunes
`distilbert/distilbert-base-cased` on the manually reviewed Stage 2 dataset.

## Labels

The model predicts BIO tags for eight entity types:

- `PERSON`
- `ORGANIZATION`
- `LOCATION`
- `TIMEDATE`
- `PRODUCT`
- `WORKOFART`
- `JOB`
- `AMOUNT`

The exact `id2label` and `label2id` mappings are stored in `config.json`.

## Training data

- Dataset:
  `Hasanblanka/polygraf-ner-stage_2-dataset`
- Dataset revision:
  `bce1f32778a333297ba1bf9749c13dd394e6256d`
- Train records: 506
- Test records: 127
- Base model revision:
  `6ea81172465e8b0ad3fddeed32b986cdcdcffcf0`

No validation split was created. The held-out test split was not used for early
stopping or hyperparameter selection.

## Training configuration

- Epochs: 4
- Batch size: 8
- Gradient accumulation: 2
- Effective batch size: 16
- Optimizer: AdamW
- Learning rate: `5e-5`
- Weight decay: `0.01`
- Warmup ratio: `0.1`
- Seed: 42
- Maximum sequence length: 512
- Truncation: disabled
- Device used for this run: CPU

## Evaluation

| Metric | Train | Test |
|---|---:|---:|
| Precision | 0.7455 | 0.5473 |
| Recall | 0.7713 | 0.6260 |
| Entity F1 | 0.7582 | 0.5840 |
| Macro F1 | 0.6735 | 0.5348 |
| Token accuracy | 0.9635 | 0.9201 |

Test results by entity type:

| Label | Precision | Recall | F1 |
|---|---:|---:|---:|
| PERSON | 0.6071 | 0.7816 | 0.6834 |
| ORGANIZATION | 0.4795 | 0.5785 | 0.5243 |
| LOCATION | 0.6398 | 0.8306 | 0.7228 |
| TIMEDATE | 0.7302 | 0.6434 | 0.6840 |
| PRODUCT | 0.0938 | 0.0732 | 0.0822 |
| WORKOFART | 0.4136 | 0.7165 | 0.5245 |
| JOB | 0.5306 | 0.3662 | 0.4333 |
| AMOUNT | 0.7358 | 0.5417 | 0.6240 |

The `metrics/` directory contains the complete machine-readable results,
benchmark, manual wild-test metrics, and test error breakdown.

## Resource measurements

- Parameters: 65,203,985
- Saved model directory: approximately 261.5 MB
- CPU throughput at batch size 1: 19.15 sequences/s
- CPU token throughput: 1,792.72 tokens/s
- Peak process RAM: approximately 639 MB
- Incremental peak RAM: approximately 244 MB

The benchmark measured 100 samples in a fresh process on an Intel Core
i5-13420H CPU. Results will vary by hardware and input length.

## Usage

```python
from transformers import pipeline

model_id = "Hasanblanka/polygraf-ner-stage3-first-model"
ner = pipeline(
    task="token-classification",
    model=model_id,
    tokenizer=model_id,
    aggregation_strategy="simple",
)

text = "Maya Ortiz joined Northstar Robotics in Lisbon on March 12, 2026."
for entity in ner(text):
    print(entity)
```

## Wild-sample QA

Ten fresh, manually annotated samples outside the main dataset were used as a
targeted stress test. Entity F1 was 0.5376. Observed problems included
excessive `WORKOFART` predictions, missed `PRODUCT` and `JOB` entities,
organization/location confusion, organization/job confusion, fragmented
multiword entities, subword fragmentation, and boundary errors.

This small manual set is useful for diagnosis but is not a statistically
representative benchmark.

## Limitations

- This is a first baseline and is not suitable for production.
- `PRODUCT` recognition is particularly weak.
- `WORKOFART` is overpredicted.
- News-heavy training data limits domain coverage.
- The project uses a custom flat, non-overlapping annotation policy.
- Nested entities are not supported.

## License and data terms

The base DistilBERT checkpoint is Apache-2.0. The fine-tuning dataset combines
sources with different terms, including evaluation-use-only starter data and
CC BY / CC BY-SA Open Newswire records. This model is therefore published with
`license: other` and is intended for evaluation and research. Users must
review and follow the source and licensing notes in the Stage 2 dataset card.