| --- |
| license: apache-2.0 |
| base_model: distilbert-base-uncased |
| tags: |
| - token-classification |
| - ner |
| - bootstrap-labels |
| - eval-mentions-bootstrap |
| metrics: |
| - seqeval |
| language: |
| - en |
| --- |
| |
| # davanstrien/eval-extraction-ner-v0 |
|
|
| Token classifier trained on **bootstrap NER labels** from [`davanstrien/eval-mentions-bootstrap`](https://huggingface.co/datasets/davanstrien/eval-mentions-bootstrap). Demonstrates the `bootstrap-labels` skill workflow: GLiNER bootstraps coarse labels, a small task-specific model is trained on them. |
|
|
| ## Training data |
|
|
| - Source: `davanstrien/eval-mentions-bootstrap` |
| - Bootstrap model: GLiNER (via `uv-scripts/gliner`) |
| - Score threshold: 0.8 (entities below this dropped) |
| - Span blacklist: ['learning_rate', 'eval_batch_size', 'epsilon', 'lr_scheduler_warmup_ratio', 'lr_scheduler_type', 'epoch', 'batch_size', 'optimizer', 'gradient_accumulation_steps', 'warmup_ratio', 'seed', 'weight_decay', 'model', 'dataset', 'transformers', 'training dataset', 'training data', 'unknown dataset', 'f1'] |
| - Train rows: 1194 |
| - Val rows: 133 |
| - Token-label distribution (excluding `O`): |
| - EVALUATION_METRIC: 7537 |
| - BENCHMARK_NAME: 3104 |
| - EVALUATION_DATASET: 1918 |
|
|
| ## Eval results |
|
|
| | Metric | Value | |
| |---|---| |
| | F1 | 0.5573 | |
| | Precision | 0.5838 | |
| | Recall | 0.5332 | |
| | Accuracy | 0.9870 | |
|
|
| (Note: held-out 10% of bootstrap labels — these are *silver labels*, not human-reviewed gold. Numbers reflect agreement with GLiNER, not absolute accuracy.) |
|
|
| ## Caveats |
|
|
| - This is a **V0 model** trained on bootstrap labels with no human review pass. Expect it to inherit GLiNER's failure modes. |
| - The intended use is *as the V1 in an active-learning loop*: deploy as Label Studio ML backend, route disagreements with GLiNER to humans, retrain on corrections. See the [bootstrap-labels skill](https://github.com/huggingface/skills) for the full workflow. |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline |
| |
| ner = pipeline("token-classification", model="davanstrien/eval-extraction-ner-v0", aggregation_strategy="simple") |
| ner("This model was evaluated on MMLU and HellaSwag.") |
| ``` |
|
|
