roberta-base-debagreement

This model is a fine-tuned version of FacebookAI/roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.8424
• Accuracy: 0.6803
• F1: 0.6776

Model description

Intended uses & limitations

This model is intended for detecting stance (agreement, disagreement, neutrality) in Reddit comment reply pairs. It was trained on political subreddit data from the DEBAGREEMENT dataset and may not generalize well to other domains or platforms.

Training and evaluation data

• Base model: FacebookAI/roberta-base (125M parameters)
• Dataset: Jiyog/debagreement-cp (DEBAGREEMENT)
• Task: 3-class sequence classification (sentence-pair input)
• Input format: body_parent (premise) + body_child (hypothesis)
• Epochs: 3
• Batch size: 16
• Max sequence length: 512 tokens
• Optimizer: AdamW (default HuggingFace Trainer)
• Weight decay: 0.01
• Best model selected by: Weighted F1

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 64
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.8592	1.0	2145	0.7596	0.6612	0.6493
0.7169	2.0	4290	0.7670	0.6780	0.6708
0.4871	3.0	6435	0.8424	0.6803	0.6776

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2