README.md

---
license: mit
language:
- de
metrics:
- f1
base_model:
- FacebookAI/xlm-roberta-base
pipeline_tag: token-classification
library_name: transformers
tags:
- arxiv:2509.07459
---

# Model Card: AIxcellent Vibes' Model for Candy Speech Detection

## Model Details

- **Model Type:** Transformer-based encoder (XLM-RoBERTa-Large)
- **Developed by:** Christian Rene Thelen, Patrick Gustav Blaneck, Tobias Bornheim, Niklas Grieger, Stephan Bialonski (FH Aachen, RWTH Aachen, ORDIX AG, Utrecht University)
- **Paper:** [AIxcellent Vibes at GermEval 2025 Shared Task on Candy Speech Detection: Improving Model Performance by Span-Level Training](https://arxiv.org/abs/2509.07459v2)
- **Base Model:** [XLM-RoBERTa-Large](https://huggingface.co/FacebookAI/xlm-roberta-large) (Conneau et al., 2020)
- **Fine-tuning Objective:** Detection of *candy speech* (positive/supportive language) in German YouTube comments.

## Model Description

This model is a fine-tuned **XLM-RoBERTa-Large** adapted for the **GermEval 2025 Shared Task on Candy Speech Detection**.
It was trained to identify *candy speech* at both:

- **Binary level:** Classify whether a comment contains candy speech.
- **Span level:** Detect the exact spans and categories of candy speech within comments, using a BIO tagging scheme across **10 categories** (positive feedback, compliment, affection declaration, encouragement, gratitude, agreement, ambiguous, implicit, group membership, sympathy).

The span-level model also proved effective for binary detection by classifying a comment as candy speech if at least one positive span was detected.

## Intended Uses

- **Research:** Analysis of positive/supportive communication in German social media.
- **Applications:** Social media analytics, conversational AI safety (mitigating sycophancy), computational social science.
- **Not for:** Deployments without fairness/robustness testing on out-of-domain data.

## Performance

- **Dataset:** 46k German YouTube comments, annotated with candy speech spans.
- **Training Data Split:** 37,057 comments (train), 9,229 (test).
- **Shared Task Results:**

  - **Subtask 1 (binary detection):** Positive F1 = **0.891** (ranked 1st)
  - **Subtask 2 (span detection):** Strict F1 = **0.631** (ranked 1st)

## Training Procedure

- **Architecture:** XLM-RoBERTa-Large + linear classification layer (BIO tagging, 21 labels including “O”).
- **Optimizer:** AdamW
- **Learning Rate:** Peak 2e-5 with linear decay and warmup (500 steps).
- **Epochs:** 20 (with early stopping).
- **Batch Size:** 32
- **Regularization:** Dropout (0.1), weight decay (0.01), gradient clipping (L2 norm 1.0).
- **Postprocessing:** BIO tag correction and subword alignment.

## Limitations

- **Domain Specificity:** Trained only on German YouTube comments; performance may degrade on other platforms, genres, or languages.
- **Overlapping Spans:** Cannot handle overlapping spans, as they were rare (<2%) in the training data.
- **Biases:** May reflect biases present in the dataset (e.g., demographic skews in YouTube communities).
- **Generalization:** Needs evaluation before deployment in real-world moderation systems.

## Ethical Considerations

- **Positive speech detection** is less studied than toxic speech, but automatic labeling of “supportiveness” may reinforce cultural biases about what counts as “positive.”
- Must be complemented with **human-in-the-loop moderation** to avoid misuse.

## Citation

If you use this model, please cite:

```
@inproceedings{thelen-etal-2025-aixcellent,
    title = "{AI}xcellent Vibes at {G}erm{E}val 2025 Shared Task on Candy Speech Detection: Improving Model Performance by Span-Level Training",
    author = "Thelen, Christian Rene  and
      Blaneck, Patrick Gustav  and
      Bornheim, Tobias  and
      Grieger, Niklas  and
      Bialonski, Stephan",
    editor = "Wartena, Christian  and
      Heid, Ulrich",
    booktitle = "Proceedings of the 21st Conference on Natural Language Processing (KONVENS 2025): Workshops",
    month = sep,
    year = "2025",
    address = "Hannover, Germany",
    publisher = "HsH Applied Academics",
    url = "https://aclanthology.org/2025.konvens-2.33/",
    pages = "398--403"
}
```

[AIxcellent Vibes at GermEval 2025 Shared Task on Candy Speech Detection: Improving Model Performance by Span-Level Training](https://aclanthology.org/2025.konvens-2.33/) (Thelen et al., KONVENS 2025)