| --- |
| license: apache-2.0 |
| pipeline_tag: token-classification |
| tags: |
| - ner |
| - gliner |
| - data-use |
| --- |
| |
| # gliner_datause_smoketest |
|
|
| Fine-tune of `urchade/gliner_large-v2.1` for data-use mention extraction |
| (dataset / survey / census / registry mentions in economics research papers). |
|
|
| ## Labels |
| - `NAMED_DATA` — a proper name, title, or acronym of a specific data source |
| - `DESCRIPTIVE_DATA` — a source described in words but not named |
| - `VAGUE_DATA` — generic data wording with no identifiable source |
|
|
| ## Training |
| - base model: `urchade/gliner_large-v2.1` |
| - dataset: `rafmacalaba/data-use-mentions` (gliner config) |
| - epochs: 1 |
| - learning rate: 5e-06 |
| - batch size: 16 |
| - precision: bf16 |
|
|
| ## Evaluation (holdout) |
|
|
| | thr | tp | fp | fn | precision | recall | f0.5 | f1 | |
| | --- | --- | --- | --- | --- | --- | --- | --- | |
| | 0.10 | 71 | 1253 | 20 | 0.0536 | 0.7802 | 0.0659 | 0.1004 | |
| | 0.20 | 54 | 519 | 37 | 0.0942 | 0.5934 | 0.1133 | 0.1627 | |
| | 0.30 | 29 | 204 | 62 | 0.1245 | 0.3187 | 0.1417 | 0.1790 | |
| | 0.40 | 21 | 67 | 70 | 0.2386 | 0.2308 | 0.2370 | 0.2346 | |
| | 0.50 | 11 | 34 | 80 | 0.2444 | 0.1209 | 0.2030 | 0.1618 | |
| | 0.60 | 4 | 23 | 87 | 0.1481 | 0.0440 | 0.1005 | 0.0678 | |
| | 0.70 | 1 | 15 | 90 | 0.0625 | 0.0110 | 0.0323 | 0.0187 | |
|
|
| **Best F0.5**: 0.2370 (thr=0.4) |
| **Best F1**: 0.2346 (thr=0.4) |
|
|
|
|
