File size: 4,568 Bytes
0f2a4bc | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 | ---
language:
- en
library_name: transformers
pipeline_tag: token-classification
tags:
- named-entity-recognition
- ner
- token-classification
- distilbert
- bio
base_model: distilbert/distilbert-base-cased
datasets:
- Hasanblanka/polygraf-ner-stage_2-dataset
license: other
model-index:
- name: Polygraf NER Stage 3 First Model
results:
- task:
type: token-classification
name: Named Entity Recognition
dataset:
name: Polygraf NER Stage 2 Expanded
type: Hasanblanka/polygraf-ner-stage_2-dataset
split: test
metrics:
- type: precision
value: 0.547275
- type: recall
value: 0.625954
- type: f1
value: 0.583976
---
# Polygraf NER Stage 3 First Model
This is the first English NER model produced for Stage 3 of the Polygraf
Applied NLP / NER technical task. It fine-tunes
`distilbert/distilbert-base-cased` on the manually reviewed Stage 2 dataset.
## Labels
The model predicts BIO tags for eight entity types:
- `PERSON`
- `ORGANIZATION`
- `LOCATION`
- `TIMEDATE`
- `PRODUCT`
- `WORKOFART`
- `JOB`
- `AMOUNT`
The exact `id2label` and `label2id` mappings are stored in `config.json`.
## Training data
- Dataset:
`Hasanblanka/polygraf-ner-stage_2-dataset`
- Dataset revision:
`bce1f32778a333297ba1bf9749c13dd394e6256d`
- Train records: 506
- Test records: 127
- Base model revision:
`6ea81172465e8b0ad3fddeed32b986cdcdcffcf0`
No validation split was created. The held-out test split was not used for early
stopping or hyperparameter selection.
## Training configuration
- Epochs: 4
- Batch size: 8
- Gradient accumulation: 2
- Effective batch size: 16
- Optimizer: AdamW
- Learning rate: `5e-5`
- Weight decay: `0.01`
- Warmup ratio: `0.1`
- Seed: 42
- Maximum sequence length: 512
- Truncation: disabled
- Device used for this run: CPU
## Evaluation
| Metric | Train | Test |
|---|---:|---:|
| Precision | 0.7455 | 0.5473 |
| Recall | 0.7713 | 0.6260 |
| Entity F1 | 0.7582 | 0.5840 |
| Macro F1 | 0.6735 | 0.5348 |
| Token accuracy | 0.9635 | 0.9201 |
Test results by entity type:
| Label | Precision | Recall | F1 |
|---|---:|---:|---:|
| PERSON | 0.6071 | 0.7816 | 0.6834 |
| ORGANIZATION | 0.4795 | 0.5785 | 0.5243 |
| LOCATION | 0.6398 | 0.8306 | 0.7228 |
| TIMEDATE | 0.7302 | 0.6434 | 0.6840 |
| PRODUCT | 0.0938 | 0.0732 | 0.0822 |
| WORKOFART | 0.4136 | 0.7165 | 0.5245 |
| JOB | 0.5306 | 0.3662 | 0.4333 |
| AMOUNT | 0.7358 | 0.5417 | 0.6240 |
The `metrics/` directory contains the complete machine-readable results,
benchmark, manual wild-test metrics, and test error breakdown.
## Resource measurements
- Parameters: 65,203,985
- Saved model directory: approximately 261.5 MB
- CPU throughput at batch size 1: 19.15 sequences/s
- CPU token throughput: 1,792.72 tokens/s
- Peak process RAM: approximately 639 MB
- Incremental peak RAM: approximately 244 MB
The benchmark measured 100 samples in a fresh process on an Intel Core
i5-13420H CPU. Results will vary by hardware and input length.
## Usage
```python
from transformers import pipeline
model_id = "Hasanblanka/polygraf-ner-stage3-first-model"
ner = pipeline(
task="token-classification",
model=model_id,
tokenizer=model_id,
aggregation_strategy="simple",
)
text = "Maya Ortiz joined Northstar Robotics in Lisbon on March 12, 2026."
for entity in ner(text):
print(entity)
```
## Wild-sample QA
Ten fresh, manually annotated samples outside the main dataset were used as a
targeted stress test. Entity F1 was 0.5376. Observed problems included
excessive `WORKOFART` predictions, missed `PRODUCT` and `JOB` entities,
organization/location confusion, organization/job confusion, fragmented
multiword entities, subword fragmentation, and boundary errors.
This small manual set is useful for diagnosis but is not a statistically
representative benchmark.
## Limitations
- This is a first baseline and is not suitable for production.
- `PRODUCT` recognition is particularly weak.
- `WORKOFART` is overpredicted.
- News-heavy training data limits domain coverage.
- The project uses a custom flat, non-overlapping annotation policy.
- Nested entities are not supported.
## License and data terms
The base DistilBERT checkpoint is Apache-2.0. The fine-tuning dataset combines
sources with different terms, including evaluation-use-only starter data and
CC BY / CC BY-SA Open Newswire records. This model is therefore published with
`license: other` and is intended for evaluation and research. Users must
review and follow the source and licensing notes in the Stage 2 dataset card.
|