yassine-mhirsi commited on
Commit
627adc4
·
verified ·
1 Parent(s): 6427b2d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +107 -3
README.md CHANGED
@@ -1,3 +1,107 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ license: mit
4
+ tags:
5
+ - stance-detection
6
+ - deberta
7
+ - text-classification
8
+ - argument-mining
9
+ metrics:
10
+ - accuracy
11
+ - f1
12
+ model-index:
13
+ - name: debertav3-stance-detection
14
+ results:
15
+ - task:
16
+ type: text-classification
17
+ name: Stance Detection
18
+ metrics:
19
+ - type: accuracy
20
+ value: 0.9997
21
+ name: Accuracy
22
+ - type: f1
23
+ value: 0.9997
24
+ name: F1 Score
25
+ ---
26
+
27
+ # Stance Detection with DeBERTa-v3-large
28
+
29
+ This model detects whether an argument supports (PRO) or opposes (CON) a given topic.
30
+
31
+ ## Model Description
32
+
33
+ - **Base Model:** microsoft/deberta-v3-large
34
+ - **Task:** Binary stance classification (PRO/CON)
35
+ - **Training Data:** IBM ArgKP-2023 dataset (~32,000 examples)
36
+ - **Calibration:** Label smoothing (0.1) for proper confidence scores
37
+
38
+ ## Performance
39
+
40
+ - **Test Accuracy:** 99.97%
41
+ - **Test F1 Score:** 99.97%
42
+ - **Mean Confidence:** 93.9% (well-calibrated)
43
+ - **Calibration:** ECE < 0.10
44
+
45
+ ## Usage
46
+
47
+ ```python
48
+ from transformers import AutoTokenizer, AutoModelForSequenceClassification
49
+ import torch
50
+
51
+ # Load model
52
+ model_name = "yassine-mhirsi/debertav3-stance-detection"
53
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
54
+ model = AutoModelForSequenceClassification.from_pretrained(model_name)
55
+
56
+ # Predict
57
+ topic = "AI should replace human teachers"
58
+ argument = "Teachers provide emotional support that AI cannot replicate"
59
+
60
+ text = f"Topic: {{topic}} [SEP] Argument: {{argument}}"
61
+ inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
62
+
63
+ with torch.no_grad():
64
+ outputs = model(**inputs)
65
+ probs = torch.nn.functional.softmax(outputs.logits, dim=-1)
66
+ predicted_class = torch.argmax(probs, dim=-1).item()
67
+
68
+ stance = "PRO" if predicted_class == 1 else "CON"
69
+ confidence = probs[0][predicted_class].item()
70
+
71
+ print(f"Stance: {{stance}}")
72
+ print(f"Confidence: {{confidence:.2%}}")
73
+ ```
74
+
75
+ ## Training Details
76
+
77
+ - **Epochs:** 3
78
+ - **Learning Rate:** 3e-6
79
+ - **Batch Size:** 4 (with gradient accumulation of 4)
80
+ - **Label Smoothing:** 0.1
81
+ - **Training Time:** ~1.5 hours on Kaggle GPU
82
+
83
+ ## Limitations
84
+
85
+ - Trained only on English argumentative text
86
+ - Best performance on formal arguments (debate-style)
87
+ - May struggle with heavy sarcasm or irony
88
+ - Calibrated for confidence, but not perfect
89
+
90
+ ## Citation
91
+
92
+ If you use this model, please cite:
93
+
94
+ ```bibtex
95
+ @misc{{stance-detection-deberta,
96
+ author = Yassine Mhirsi,
97
+ title = {{Stance Detection with DeBERTa-v3-large}},
98
+ year = {{2025}},
99
+ publisher = {{Hugging Face}},
100
+ howpublished = {{\\url{{https://huggingface.co/yassine-mhirsi/debertav3-stance-detection}}}}
101
+ }}
102
+ ```
103
+
104
+ ## License
105
+
106
+ MIT License
107
+ ---