Christian Rene Thelen commited on
Commit
1e67da4
·
1 Parent(s): b475e1c

Added model card informatio and citation

Browse files
Files changed (1) hide show
  1. README.md +86 -1
README.md CHANGED
@@ -8,4 +8,89 @@ base_model:
8
  - FacebookAI/xlm-roberta-base
9
  pipeline_tag: token-classification
10
  library_name: transformers
11
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
  - FacebookAI/xlm-roberta-base
9
  pipeline_tag: token-classification
10
  library_name: transformers
11
+ tags:
12
+ - arxiv:2509.07459
13
+ ---
14
+
15
+ # Model Card: AIxcellent Vibes' Model for Candy Speech Detection
16
+
17
+ ## Model Details
18
+
19
+ - **Model Type:** Transformer-based encoder (XLM-RoBERTa-Large)
20
+ - **Developed by:** Christian Rene Thelen, Patrick Gustav Blaneck, Tobias Bornheim, Niklas Grieger, Stephan Bialonski (FH Aachen, RWTH Aachen, ORDIX AG, Utrecht University)
21
+ - **Paper:** [AIxcellent Vibes at GermEval 2025 Shared Task on Candy Speech Detection: Improving Model Performance by Span-Level Training](https://arxiv.org/abs/2509.07459v2)
22
+ - **Base Model:** [XLM-RoBERTa-Large](https://huggingface.co/FacebookAI/xlm-roberta-large) (Conneau et al., 2020)
23
+ - **Fine-tuning Objective:** Detection of *candy speech* (positive/supportive language) in German YouTube comments.
24
+
25
+ ## Model Description
26
+
27
+ This model is a fine-tuned **XLM-RoBERTa-Large** adapted for the **GermEval 2025 Shared Task on Candy Speech Detection**.
28
+ It was trained to identify *candy speech* at both:
29
+
30
+ - **Binary level:** Classify whether a comment contains candy speech.
31
+ - **Span level:** Detect the exact spans and categories of candy speech within comments, using a BIO tagging scheme across **10 categories** (positive feedback, compliment, affection declaration, encouragement, gratitude, agreement, ambiguous, implicit, group membership, sympathy).
32
+
33
+ The span-level model also proved effective for binary detection by classifying a comment as candy speech if at least one positive span was detected.
34
+
35
+ ## Intended Uses
36
+
37
+ - **Research:** Analysis of positive/supportive communication in German social media.
38
+ - **Applications:** Social media analytics, conversational AI safety (mitigating sycophancy), computational social science.
39
+ - **Not for:** Deployments without fairness/robustness testing on out-of-domain data.
40
+
41
+ ## Performance
42
+
43
+ - **Dataset:** 46k German YouTube comments, annotated with candy speech spans.
44
+ - **Training Data Split:** 37,057 comments (train), 9,229 (test).
45
+ - **Shared Task Results:**
46
+
47
+ - **Subtask 1 (binary detection):** Positive F1 = **0.891** (ranked 1st)
48
+ - **Subtask 2 (span detection):** Strict F1 = **0.631** (ranked 1st)
49
+
50
+ ## Training Procedure
51
+
52
+ - **Architecture:** XLM-RoBERTa-Large + linear classification layer (BIO tagging, 21 labels including “O”).
53
+ - **Optimizer:** AdamW
54
+ - **Learning Rate:** Peak 2e-5 with linear decay and warmup (500 steps).
55
+ - **Epochs:** 20 (with early stopping).
56
+ - **Batch Size:** 32
57
+ - **Regularization:** Dropout (0.1), weight decay (0.01), gradient clipping (L2 norm 1.0).
58
+ - **Postprocessing:** BIO tag correction and subword alignment.
59
+
60
+ ## Limitations
61
+
62
+ - **Domain Specificity:** Trained only on German YouTube comments; performance may degrade on other platforms, genres, or languages.
63
+ - **Overlapping Spans:** Cannot handle overlapping spans, as they were rare (<2%) in the training data.
64
+ - **Biases:** May reflect biases present in the dataset (e.g., demographic skews in YouTube communities).
65
+ - **Generalization:** Needs evaluation before deployment in real-world moderation systems.
66
+
67
+ ## Ethical Considerations
68
+
69
+ - **Positive speech detection** is less studied than toxic speech, but automatic labeling of “supportiveness” may reinforce cultural biases about what counts as “positive.”
70
+ - Must be complemented with **human-in-the-loop moderation** to avoid misuse.
71
+
72
+ ## Citation
73
+
74
+ If you use this model, please cite:
75
+
76
+ ```
77
+ @inproceedings{thelen-etal-2025-aixcellent,
78
+ title = "{AI}xcellent Vibes at {G}erm{E}val 2025 Shared Task on Candy Speech Detection: Improving Model Performance by Span-Level Training",
79
+ author = "Thelen, Christian Rene and
80
+ Blaneck, Patrick Gustav and
81
+ Bornheim, Tobias and
82
+ Grieger, Niklas and
83
+ Bialonski, Stephan",
84
+ editor = "Wartena, Christian and
85
+ Heid, Ulrich",
86
+ booktitle = "Proceedings of the 21st Conference on Natural Language Processing (KONVENS 2025): Workshops",
87
+ month = sep,
88
+ year = "2025",
89
+ address = "Hannover, Germany",
90
+ publisher = "HsH Applied Academics",
91
+ url = "https://aclanthology.org/2025.konvens-2.33/",
92
+ pages = "398--403"
93
+ }
94
+ ```
95
+
96
+ [AIxcellent Vibes at GermEval 2025 Shared Task on Candy Speech Detection: Improving Model Performance by Span-Level Training](https://aclanthology.org/2025.konvens-2.33/) (Thelen et al., KONVENS 2025)