metadata
language: en
license: mit
tags:
- stance-detection
- text-classification
- argument-mining
- deberta-v3
metrics:
- accuracy
- f1
model-index:
- name: debertav3-stance-detection
results:
- task:
type: text-classification
name: Stance Detection
metrics:
- type: accuracy
value: 0.9997
name: Accuracy
- type: f1
value: 0.9997
name: F1 Score
base_model:
- microsoft/deberta-v3-large
pipeline_tag: text-classification
datasets:
- NLP-Debater-Project/IBM-Debater-ArgKP
Stance Detection with DeBERTa-v3-large
This model detects whether an argument supports (PRO) or opposes (CON) a given topic.
Model Description
- Base Model: microsoft/deberta-v3-large
- Task: Binary stance classification (PRO/CON)
- Training Data: IBM ArgKP-2023 dataset (~32,000 examples)
- Calibration: Label smoothing (0.1) for proper confidence scores
Performance
- Test Accuracy: 99.97%
- Test F1 Score: 99.97%
- Mean Confidence: 93.9% (well-calibrated)
- Calibration: ECE < 0.10
Usage
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
# Load model
model_name = "yassine-mhirsi/debertav3-stance-detection"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
# Predict
topic = "AI should replace human teachers"
argument = "Teachers provide emotional support that AI cannot replicate"
text = f"Topic: {{topic}} [SEP] Argument: {{argument}}"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
outputs = model(**inputs)
probs = torch.nn.functional.softmax(outputs.logits, dim=-1)
predicted_class = torch.argmax(probs, dim=-1).item()
stance = "PRO" if predicted_class == 1 else "CON"
confidence = probs[0][predicted_class].item()
print(f"Stance: {{stance}}")
print(f"Confidence: {{confidence:.2%}}")
Training Details
- Epochs: 3
- Learning Rate: 3e-6
- Batch Size: 4 (with gradient accumulation of 4)
- Label Smoothing: 0.1
- Training Time: ~1.5 hours on Kaggle GPU
Limitations
- Trained only on English argumentative text
- Best performance on formal arguments (debate-style)
- May struggle with heavy sarcasm or irony
- Calibrated for confidence, but not perfect
Citation
If you use this model, please cite:
@misc{{stance-detection-deberta,
author = Yassine Mhirsi,
title = {{Stance Detection with DeBERTa-v3-large}},
year = {{2025}},
publisher = {{Hugging Face}},
howpublished = {{\\url{{https://huggingface.co/yassine-mhirsi/debertav3-stance-detection}}}}
}}