DeBERTa-v3-large-clause-metaphor

This model is a fine-tuned version of Microsoft DeBERTa-v3-large on the VUA-20 (VUAMC) dataset for binary metaphor detection in English at token level. Training uses clause-level segmentation (SpaCy: ROOT + advcl/ccomp/relcl) and a two-class scheme (non-metaphor / metaphor), following the MIPVU annotation guidelines (Steen et al. 2010).

Model description

• Base model: microsoft/deberta-v3-large
• Task: Token classification (binary metaphor detection)
• Training unit: Clauses (not full sentences), so each example is one clause with aligned word-level labels.

Training & evaluation data

• Dataset: VUA-20 (VUAMC), same underlying corpus as VUAM (Steen et al. 2010).
• Data prep: CSV files from VUA-20/VUAMC → SpaCy annotation (PTB POS, dependency) → clause splitting → train/val/test JSON. Test set: 10 held-out fragments (29,241 tokens).

Training hyperparameters

Parameter	Value
Batch size	2
Gradient accumulation	8 (effective batch 16)
Epochs	4
Learning rate	2e-5
Max sequence length	192
Optimizer / scheduler	Default Trainer (AdamW, linear decay)

Results

Evaluation on the held-out test set (10 files, 29,241 tokens):

Metric	Value
F1	75.83
Precision	78.08
Recall	73.69

Validation F1 during training (last epoch): ~79.05.

By part-of-speech (selected)

POS	n	F1
IN	2654	87.87
DT	2392	90.87
RP	108	78.57
RB	1531	68.57
Other (aggregate)	22556	67.16

Label dictionary

This model uses two labels (binary token classification):

{
  "LABEL_0": "non_metaphor",
  "LABEL_1": "metaphor"
}

Subwords are aligned to words via the tokenizer’s word_ids; the first subword of each word is used for prediction.

Usage example

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

model_path = "your-org/deberta-v3-large-clause-metaphor"  # or local path
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForTokenClassification.from_pretrained(model_path)

words = ["The", "government", "attacked", "the", "proposal", "."]
inputs = tokenizer(
    [words],
    is_split_into_words=True,
    return_tensors="pt",
    truncation=True,
    max_length=192,
)
word_ids = inputs.word_ids(batch_index=0)
with torch.no_grad():
    logits = model(**inputs).logits
preds = logits.argmax(dim=-1)[0].tolist()

# Map subwords back to words (first subword per word)
word_predictions = {}
for i, wid in enumerate(word_ids):
    if wid is not None and wid not in word_predictions:
        word_predictions[wid] = 1 if preds[i] == 1 else 0
for i, w in enumerate(words):
    label = "metaphor" if word_predictions.get(i, 0) == 1 else "non_metaphor"
    print(f"{w}\t{label}")

Citation

If you use this model, please cite the VUA/MIPVU dataset, the base model, and this model (author below).

Model author: Tommy Leo — 1683619168tl@gmail.com

Dataset (MIPVU / VUA):

@book{steen2010method,
  title={A method for linguistic metaphor identification: From MIP to MIPVU},
  author={Steen, Gerard and Dorst, Lettie and Herrmann, J. and Kaal, Anna and Krennmayr, Tina and Pasma, Trijntje},
  volume={14},
  year={2010},
  publisher={John Benjamins Publishing}
}

Base model: Microsoft DeBERTa.

This model: If you use this model, please cite:

@misc{leo2025debertav3largeclausemetaphor,
  title={DeBERTa-v3-large-clause-metaphor: Metaphor detection at clause level},
  author={Leo, Tommy},
  year={2025},
  howpublished={\url{https://huggingface.co/tommyleo2077/deberta-v3-large-clause-metaphor}},
  note={Contact: 1683619168tl@gmail.com}
}