mariadelcarmenramirez/metaphor-cat-mdeberta-no-weights

metaphor-cat-mdberta-no-weights

This model is a fine-tuned version of microsoft/mdeberta-v3-base on the Catalan metaphor detection dataset metaphor-catalan.

It achieves the following results on the evaluation set:

• Loss: 0.1220
• Precision: 0.75
• Recall: 0.4286
• F1: 0.5455
• Accuracy: 0.9641

Model description

This model is a DeBERTa-based transformer fine-tuned for token-level metaphor detection in Catalan. It predicts whether each token in a sentence belongs to a metaphorical expression. The model is suitable for research and applications in computational linguistics, natural language understanding, and figurative language detection in Catalan text.

Intended uses & limitations

Intended uses:

• Detecting metaphors in Catalan text.
• Assisting linguistic research on figurative language.
• Integrating metaphor detection into NLP pipelines for Catalan text analysis.

Limitations:

• The dataset is relatively small and may not cover all domains or metaphorical expressions.
• Recall is moderate (0.4286), meaning the model may miss some metaphors.
• The dataset is imbalanced, with many more literal tokens than metaphorical ones.
• Not suitable for critical decision-making applications without further evaluation.

Training and evaluation data

Training dataset: metaphor-catalan

• Contains Catalan sentences annotated for metaphor detection at the token level.

Training procedure

Hyperparameters

• Learning rate: 2e-05
• Train batch size: 8
• Evaluation batch size: 8
• Optimizer: AdamW (fused, betas=(0.9,0.999), epsilon=1e-08)
• LR scheduler: linear
• Epochs: 5
• Mixed precision: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	1.0	66	0.1620	0.0	0.0	0.0	0.9498
0.2683	2.0	132	0.1288	0.6471	0.2619	0.3729	0.9557
0.2683	3.0	198	0.1193	0.75	0.4286	0.5455	0.9641
0.0938	4.0	264	0.1242	0.7391	0.4048	0.5231	0.9629
0.0545	5.0	330	0.1220	0.75	0.4286	0.5455	0.9641

Framework versions

• Transformers: 4.57.3
• PyTorch: 2.9.0+cu126
• Datasets: 4.0.0
• Tokenizers: 0.22.1