README.md

---
library_name: transformers
license: apache-2.0
base_model: allenai/specter2_base
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: results
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# 📗 SPECTER2–MAG (Multiclass Classification on MAG Level-0 Fields of Study)

This model is a fine-tuned version of [allenai/specter2_base](https://huggingface.co/allenai/specter2_base) for multiclass bibliometric classification using MAG Fields of Study – Level 0 (SciDocs).
It achieves the following results on the evaluation set:
- Loss: 1.0598
- Accuracy: 0.8310
- Precision Micro: 0.8310
- Precision Macro: 0.8290
- Recall Micro: 0.8310
- Recall Macro: 0.8276
- F1 Micro: 0.8310
- F1 Macro: 0.8263

## Model description

This model is a fine-tuned version of SPECTER2 (`allenai/specter2_base`) adapted for multiclass classification across the 19 top-level Fields of Study (FoS) from the Microsoft Academic Graph (MAG).

The model accepts the title, abstract, or title + abstract of a scientific publication and assigns it to exactly one of the MAG Level-0 domains (e.g., Biology, Chemistry, Computer Science, Engineering, Psychology).

Key characteristics:
* Base model: allenai/specter2_base
* Task: multiclass document classification
* Labels: 19 MAG Field of Study Level-0 categories
* Activation: softmax
* Loss: CrossEntropyLoss
* Output: single best-matching FoS category

MAG Level-0 represents broad disciplinary domains designed for high-level categorization of scientific documents.

## Intended uses & limitations

### Intended uses
This multiclass MAG model is suitable for:

- Assigning publications to **top-level scientific disciplines**
- Enriching metadata in:
  - repositories  
  - research output systems  
  - funding and project datasets  
  - bibliometric dashboards  
- Supporting scientometric analyses such as:
  - broad-discipline portfolio mapping  
  - domain-level clustering  
  - modeling research diversification  
- Classifying documents when only **title/abstract** is available

The model supports inputs such as:
- **title only**
- **abstract only**
- **title + abstract** (recommended)

### Limitations
- MAG Level-0 categories are **very coarse** (e.g., *Biology*, *Medicine*, *Engineering*), and do not represent subfields.
- Documents spanning multiple fields must be forced into **one** label—an inherent limitation of multiclass classification.
- The training labels come from **MAG’s automatic field assignment pipeline**, not manual expert annotation.
- Not suitable for:
  - fine-grained subdisciplines  
  - downstream tasks requiring multilabel outputs  
  - WoS Categories or ASJC Areas (use separate models)  
  - clinical or regulatory decision-making  

Predictions should be treated as **high-level disciplinary metadata**, not detailed field classification.

## Training and evaluation data

### Source dataset: **SciDocs**

Training data comes from the **SciDocs** dataset, introduced together with the original SPECTER paper:

> **SciDocs** provides citation graphs, titles, abstracts, and **MAG Fields of Study** for scientific documents derived from MAG.  
> For this model, we use **MAG Level-0 FoS**, the 19 top-level scientific domains.

Dataset characteristics:

| Property | Value |
|---------|-------|
| Documents | ~40k scientific papers |
| Labels | 19 FoS Level-0 categories |
| Input fields | Abstract |
| Task type | Multiclass |
| Source | SciDocs (SPECTER paper) |
| License | CC-BY |

## Training procedure

### Preprocessing
- Input text constructed as:  
  `abstract`
- Tokenization using the SPECTER2 tokenizer  
- Maximum sequence length: **512 tokens**

### Model
- Base model: `allenai/specter2_base`  
- Classification head: linear layer → softmax  
- Loss: **CrossEntropyLoss**  

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 10

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Precision Micro | Precision Macro | Recall Micro | Recall Macro | F1 Micro | F1 Macro |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:---------------:|:---------------:|:------------:|:------------:|:--------:|:--------:|
| 0.2603        | 1.0   | 1094 | 0.6733          | 0.8243   | 0.8243          | 0.8315          | 0.8243       | 0.8198       | 0.8243   | 0.8222   |
| 0.1779        | 2.0   | 2188 | 0.6955          | 0.8240   | 0.8240          | 0.8198          | 0.8240       | 0.8203       | 0.8240   | 0.8176   |
| 0.1628        | 3.0   | 3282 | 0.8130          | 0.8315   | 0.8315          | 0.8296          | 0.8315       | 0.8265       | 0.8315   | 0.8269   |
| 0.1136        | 4.0   | 4376 | 0.9842          | 0.8227   | 0.8227          | 0.8254          | 0.8227       | 0.8192       | 0.8227   | 0.8205   |
| 0.0666        | 5.0   | 5470 | 1.0598          | 0.8310   | 0.8310          | 0.8290          | 0.8310       | 0.8276       | 0.8310   | 0.8263   |

### Evaluation results

|                       |   precision |   recall |   f1-score |     support |
|:----------------------|------------:|---------:|-----------:|------------:|
| Art                   |    0.654867 | 0.845714 |   0.738155 |  175        |
| Biology               |    0.982222 | 0.973568 |   0.977876 |  227        |
| Business              |    0.914894 | 0.877551 |   0.895833 |  196        |
| Chemistry             |    0.97449  | 0.969543 |   0.97201  |  197        |
| Computer science      |    0.960452 | 0.894737 |   0.926431 |  190        |
| Economics             |    0.816425 | 0.782407 |   0.799054 |  216        |
| Engineering           |    0.906103 | 0.927885 |   0.916865 |  208        |
| Environmental science |    0.975369 | 0.916667 |   0.945107 |  216        |
| Geography             |    0.758454 | 0.912791 |   0.828496 |  172        |
| Geology               |    0.96729  | 0.976415 |   0.971831 |  212        |
| History               |    0.62987  | 0.518717 |   0.568915 |  187        |
| Materials science     |    0.932432 | 0.958333 |   0.945205 |  216        |
| Mathematics           |    0.938776 | 0.94359  |   0.941176 |  195        |
| Medicine              |    0.982558 | 0.923497 |   0.952113 |  183        |
| Philosophy            |    0.752874 | 0.748571 |   0.750716 |  175        |
| Physics               |    0.964824 | 0.974619 |   0.969697 |  197        |
| Political science     |    0.642512 | 0.661692 |   0.651961 |  201        |
| Psychology            |    0.806283 | 0.758621 |   0.781726 |  203        |
| Sociology             |    0.438889 | 0.427027 |   0.432877 |  185        |
| accuracy              |    0.845641 | 0.845641 |   0.845641 |    0.845641 |
| macro avg             |    0.842083 | 0.841681 |   0.840318 | 3751        |
| weighted avg          |    0.847843 | 0.845641 |   0.845311 | 3751        |


### Framework versions

- Transformers 4.57.1
- Pytorch 2.8.0+cu126
- Datasets 3.6.0
- Tokenizers 0.22.1