| --- |
| language: |
| - en |
| library_name: transformers |
| license: apache-2.0 |
| pipeline_tag: text-classification |
| tags: |
| - bibr |
| - OECD |
| - scientific-paper-classification |
| - MiniLM |
| --- |
| |
| # bibr OECD/paper-type classifier — DeepSeek teacher, 2026-07-19 |
|
|
| This is the DeepSeek-v4-Flash-teacher MiniLM candidate for bibr's multitask |
| scientific-paper classifier. It predicts OECD Level 1, OECD Level 2, and paper |
| type from title plus abstract. |
|
|
| ## Architecture |
|
|
| - Encoder: `sentence-transformers/all-MiniLM-L6-v2` |
| - Input template: `title_abstract_v1` |
| - Maximum input length: 256 tokens |
| - Heads: OECD L1, OECD L2, paper type |
| - Model SHA-256: |
| `7a46c595cf3bb8e1eff39303786e8b7d16c95ea87f9f06b12d4a753f0be1e580` |
|
|
| ## Training provenance |
|
|
| - Training corpus rows: 122,363 |
| - Deterministic split: 91,771 train / 12,237 validation / 18,355 test |
| - DeepSeek teacher changed 16,220 L1 labels relative to the matched OpenAlex |
| baseline. |
| - Teacher-labelled rows are training supervision, not evaluation gold. |
|
|
| ## Evaluation |
|
|
| On the frozen 1,500-row Phase-A panel: |
|
|
| | Model | Accuracy | Macro-F1 | |
| |---|---:|---:| |
| | Matched OpenAlex MiniLM baseline | 0.7333 | 0.7280 | |
| | This checkpoint | **0.8120** | **0.8096** | |
| | Previously shipped SPECTER2 checkpoint | 0.6887 | 0.6874 | |
|
|
| The matched MiniLM gain was +0.0787 accuracy and +0.0816 macro-F1. The paired |
| correctness table contained 158 teacher-only correct rows and 40 baseline-only |
| correct rows; exact McNemar p-value was `8.687e-18`. |
|
|
| The Phase-A reference is explicitly |
| `provisional_unadjudicated_codex_panel`. It is not human-adjudicated gold, so |
| these figures support candidate selection but not a final scientific-quality |
| claim. |
|
|
| ## Bundle contract |
|
|
| The bibr loader requires: |
|
|
| - `model.safetensors` |
| - `tokenizer.json` |
| - `tokenizer_config.json` |
| - `label_maps.json` |
| - `inference_config.json` |
|
|
| `test_metrics.json`, `test_l1_diagnostics.json`, and `phaseA_metrics.json` |
| provide training and provisional-panel diagnostics. |
|
|
