THemidli/applied-ner-stage4-bert-tiny-improved

Applied NER Stage 4 — Improved BERT Tiny

An eight-label English token classifier fine-tuned from google/bert_uncased_L-2_H-128_A-2. Repository: THemidli/applied-ner-stage4-bert-tiny-improved.

Results

Exact entity-level seqeval metrics:

Split	Precision	Recall	F1	Token accuracy
Train	0.9540	0.9709	0.9624	0.9948
Test	0.4261	0.5264	0.4710	0.8332

Label	Precision	Recall	F1	Support
PERSON	0.487	0.651	0.557	195
ORGANIZATION	0.216	0.252	0.233	147
LOCATION	0.436	0.545	0.484	143
TIMEDATE	0.785	0.832	0.808	167
PRODUCT	0.168	0.181	0.174	127
WORKOFART	0.136	0.247	0.176	97
JOB	0.664	0.798	0.725	99
AMOUNT	0.540	0.587	0.562	104

On 40 fresh, manually gold-labeled wild probes, exact span F1 was 0.5849 (precision 0.5439, recall 0.6327). Test F1 changed by +0.0025 versus Stage 3.

Training

• Dataset: THemidli/applied-ner-stage4-improved
• Seed: 20260802
• Hardware: Apple MPS (macOS-27.0-arm64-arm-64bit)
• Runtime: 14.937 seconds
• Records/chunks: 841/865 train; 159/165 test
• Maximum length: 256; fast-tokenizer overflow chunks, no overlapping stride
• Hyperparameters: {"attention_dropout": 0.1, "classifier_dropout": 0.1, "epochs": 16, "eval_batch_size": 64, "hidden_dropout": 0.1, "label_smoothing_factor": 0.0, "learning_rate": 0.0005, "scheduler": "linear", "train_batch_size": 32, "warmup_steps": 45, "weight_decay": 0.02}
• No validation split and no test-driven checkpoint selection

Footprint and CPU benchmark

• Parameters: 4,371,601 (17.49 MB tensor storage)
• Saved artifact: 18.21 MB
• Model-load RSS delta: 31.82 MB
• End-to-end inference RSS delta: 41.48 MB
• CPU throughput: 11610.4 examples/s at batch 32 with 8 threads
• Mean latency: 0.0861 ms/example at that batch size

The benchmark covers tokenizer plus PyTorch CPU forward pass over 40 short probes, repeated 50 times. It is workload- and hardware-specific, not single-request latency.

Labels

PERSON, ORGANIZATION, LOCATION, TIMEDATE, PRODUCT, WORKOFART, JOB, AMOUNT using BIO encoding.

Limitations

This is a 4.37M-parameter uncased two-layer BERT trained on a small, heterogeneous dataset. It is a compact baseline, not a production privacy system. Rare works/products, company-versus-product context, exact boundaries, and subword-heavy names remain weak. The 40-probe wild set is diagnostic, not a population benchmark.