metaphor-cat-roberta-large-no-weights

This model is a fine-tuned version of projecte-aina/roberta-large-ca-v2 on the Catalan metaphor detection dataset metaphor-catalan.

It achieves the following results on the evaluation set:

• Loss: 0.1317
• Precision: 0.9130
• Recall: 0.5
• F1: 0.6462
• Accuracy: 0.9725

Model description

This model is a RoBERTa-based transformer fine-tuned to classify Catalan sentences as containing metaphors or not. It is suitable for research and applications in computational linguistics, natural language understanding, and metaphor detection in Catalan text.

Intended uses & limitations

Intended uses:

• Detecting metaphors in Catalan text.
• Assisting in linguistic research or educational tools.
• Integrating into NLP pipelines for text analysis in Catalan.

Limitations:

• The dataset is relatively small and may not cover all domains or metaphorical expressions.
• Recall is moderate (0.5), which means the model may miss some metaphors.
• Not suitable for critical decision-making applications without further evaluation.

Training and evaluation data

Training dataset: metaphor-catalan

• Contains sentences labeled as metaphorical or literal in Catalan.

Training procedure

Hyperparameters

• Learning rate: 2e-05
• Train batch size: 4
• Evaluation batch size: 4
• Total effective batch size: 8 (gradient accumulation 2 steps)
• Optimizer: AdamW (fused, betas=(0.9,0.999), epsilon=1e-08)
• LR scheduler: linear
• Epochs: 5
• Mixed precision: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	1.0	66	0.1558	0.0	0.0	0.0	0.9498
0.2326	2.0	132	0.1153	0.76	0.4524	0.5672	0.9653
0.2326	3.0	198	0.1102	0.9130	0.5	0.6462	0.9725
0.0765	4.0	264	0.1257	0.88	0.5238	0.6567	0.9725
0.0322	5.0	330	0.1317	0.9130	0.5	0.6462	0.9725

Framework versions

• Transformers: 4.57.3
• PyTorch: 2.9.0+cu126
• Datasets: 4.0.0
• Tokenizers: 0.22.1