README.md

---
license: mit
library_name: transformers
pipeline_tag: text-classification
base_model: microsoft/deberta-v3-large
tags:
- citation-function-classification
- scholarly-positioning
- related-work-generation
- rwgbench
- multicite
datasets:
- multicite
metrics:
- f1
---

# RWGBench Citation Frame Classifier

This model is the citation-frame classifier used by RWGBench to compute
Citation Frame Alignment (CFA). CFA evaluates whether a generated related work
section uses citations with a rhetorical frame distribution similar to the
author-written reference section.

The classifier is a DeBERTa-v3-large sequence-classification model fine-tuned
for multi-label citation-function prediction. It predicts seven
MultiCite-derived labels:

| Label | Intended citation function |
|---|---|
| `motivation` | motivates the target problem or research direction |
| `background` | provides general background or prior findings |
| `uses` | describes methods, data, tools, or results used by the target work |
| `extends` | indicates that the target work builds on or extends prior work |
| `similarities` | marks similarity or close relationship to prior work |
| `differences` | marks contrast, limitation, gap, or distinction |
| `future_work` | discusses future directions |

## Intended Use

The model is intended for automatic evaluation and analysis of related work
generation systems. In RWGBench, citation contexts are extracted from generated
and reference related work sections, classified with this model, aggregated
into frame-count profiles, and compared with macro count F1.

It is not intended as a general-purpose judge of paper quality or citation
correctness. It should be used together with citation-selection and
citation-support metrics.

## Files

| File | Description |
|---|---|
| `config.json`, `model.safetensors` | Transformers sequence-classification checkpoint |
| `tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json`, `spm.model`, `added_tokens.json` | Tokenizer files |
| `thresholds.json` | Per-label development thresholds used for multi-label decoding |
| `metrics.json` | Development and held-out MultiCite evaluation metrics |
| `label_map.json` | Label mapping used during training |
| `rwgb_classifier_config.json` | RWGBench-specific usage metadata |

## Evaluation

Held-out MultiCite test performance:

| Metric | Score |
|---|---:|
| Macro F1 | 0.5585 |
| Micro F1 | 0.6751 |
| Samples F1 | 0.6933 |
| Weighted F1 | 0.6750 |
| Subset accuracy | 0.5119 |

Per-label held-out F1:

| Label | F1 |
|---|---:|
| `motivation` | 0.3523 |
| `background` | 0.7756 |
| `uses` | 0.7372 |
| `extends` | 0.4450 |
| `similarities` | 0.5508 |
| `differences` | 0.6274 |
| `future_work` | 0.4211 |

The thresholds in `thresholds.json` are applied after sigmoid probabilities.
If no label passes its threshold, RWGBench assigns the highest-probability label
so that every citation context contributes at least one frame.

## Usage With RWGBench

Download the model into the RWGBench code repository:

```text
models/citation_frame_multicite/
  config.json
  model.safetensors
  tokenizer.json
  thresholds.json
  ...
```

run:

```bash
python src/evaluation/evaluate_generated_results.py \
  --input results/your_results.json \
  --output results/your_eval.json \
  --gold data/gold100_papers.json \
  --corpus data/corpus.json \
  --citation-frame-model-path models/citation_frame_multicite
```

RWGBench automatically marks numbered citations with `<cite>...</cite>` before
classification, matching the input format used during training.