File size: 4,644 Bytes
c24bfeb | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 | # Denotational Type Classifier
This model classifies the denotational relation between two word senses, represented as a pair of definitions. It is based on `roberta-large` with a sequence-classification head and is trained for lexical-semantic relation classification.
The model predicts one of the following relation types:
* `generalization`
* `specialization`
* `metaphor`
* `metonymy`
* `homonymy`
* `antonymy`
The first five labels correspond to the denotational relation types evaluated in SenseRel. `antonymy` is included because it is present in the fine-tuning data, although it is not part of the expert-annotated SenseRel denotational test set.
## Model Details
* **Model type:** Causal language model
* **Base model:** `meta-llama/Meta-Llama-3.1-8B`
* **Task:** Denotational relation classification via text generation
* **Language:** English
## Intended Use
This model is intended for research on lexical semantics, semantic change, polysemy, and sense-level semantic relations. It can be used to classify the relation between two definitions of a word sense, for example whether a newer meaning is a generalization, specialization, metaphorical extension, metonymic extension, homonym, or antonymic development of another meaning.
Example use cases include:
* studying semantic change in dictionaries or lexical resources;
* analyzing polysemous word senses;
* supporting sense-level semantic annotation;
* scaling exploratory studies of denotational change.
The model is not intended for high-stakes decision-making or for general-purpose natural language understanding outside lexical-semantic relation classification.
## Input Format
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_id = "ChangeIsKey/denotational-llama-classifier"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
definition_1 = "the young of the domestic cow"
definition_2 = "the young of various large mammals"
text = f"{definition_1} <|s|> {definition_2} <|t|>"
inputs = tokenizer(text, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
next_token_logits = outputs.logits[0, -1, :]
label_tokens = [
"<|generalization|>",
"<|specialization|>",
"<|metaphor|>",
"<|metonymy|>",
"<|homonymy|>",
"<|antonymy|>",
]
label_token_ids = tokenizer.convert_tokens_to_ids(label_tokens)
label_logits = next_token_logits[label_token_ids]
predicted_id = label_token_ids[torch.argmax(label_logits).item()]
prediction = tokenizer.convert_ids_to_tokens(predicted_id)
print(prediction)
```
## Training Data
The model was fine-tuned on a combined denotational-relation dataset referred to as `WN+CN+UM`, constructed from existing lexical-semantic resources:
* ChainNet for metaphor, metonymy, and homonymy relations;
* UniMet for additional metonymy examples;
* WordNet for generalization, specialization, and auto-antonymy examples.
These resources use definition or synset-pair information as proxies for relations between word senses.
## Training Procedure
The base Llama model was fine-tuned using a causal language modeling objective.
Each training instance consists of a prompt containing two dictionary definitions followed by the target relation label. During fine-tuning, the model learns to generate the correct label given the input definitions.
## Evaluation
The model was evaluated on:
1. the `WN+CN+UM` test set;
2. the SenseRel expert-annotated denotational dataset.
| Dataset | Weighted F1 |
| ----------------------------- | ----------: |
| WN+CN+UM test set | 0.737 |
| SenseRel denotational dataset | 0.658 |
On the WN+CN+UM test set, this model achieved the best reported score among the evaluated systems.
## Citation
If you use this model, please cite:
```bibtex
@inproceedings{cassotti-etal-2026-senserel,
title = "{S}ense{R}el: A Sense-Level Benchmark for Denotational and Connotational Meaning Relations",
author = "Cassotti, Pierluigi and
Baes, Naomi and
De Pascale, Stefano and
de S{\'a}, J{\'a}der Martins Camboim and
Periti, Francesco and
Haslam, Nick and
Geeraerts, Dirk and
Tahmasebi, Nina",
booktitle = "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
month = jul,
year = "2026",
address = "San Diego, California, United States",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.acl-long.20/",
pages = "499--515"
}
```
|