File size: 5,201 Bytes
178b774 d6e8792 178b774 d6e8792 178b774 d6e8792 178b774 d6e8792 178b774 d6e8792 178b774 d6e8792 178b774 d6e8792 178b774 d6e8792 178b774 d6e8792 178b774 7e2962a d6e8792 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 | ---
license: cc-by-nc-4.0
tags:
- social-media-analysis
- compulsion-detection
- political-tweets
- bayesian-classifier
- digital-phenotyping
- toxicity-index
pipeline_tag: text-classification
---
# X-Box Compulsion & Toxicity Index Classifier
Bayesian temporal phenotyping + 12-head text classification pipeline for detecting
compulsive social media usage patterns and computing the Toxicity Index (TI) for
political Twitter/X accounts.
## Architecture
**Temporal Model**: Calibrated logistic regression on 5 compulsion signatures
(burstiness, time-of-day entropy, Hawkes self-excitation, night intensity, weekend ratio).
**Text Classification**: 12 heads producing the per-tweet Toxicity Index.
**Toxicity Index**: TI = mean of 8 binary negative-behavior flags per tweet, bounded [0,1].
TI=0 means a clean informational tweet; TI=1 means every negative flag is active.
## Validation
**Compulsion Model** (n=32, independent ground truth):
- Spearman r = 0.912 (permutation p=0.001, bootstrap 95% CI [0.845, 0.965])
- AUC = 0.933 (permutation p=0.003, bootstrap 95% CI [0.928, 1.000])
- Repeated 5-fold (x20): AUC = 0.953 +/- 0.076
- Brier score: 0.101
**Text Classification Label Reliability** (test-retest, n=75):
- Ragebait: Pearson r=0.889, Cohen kappa=0.479
- Tribal signal: Pearson r=0.862, Cohen kappa=0.730
- Performative outrage: Pearson r=0.777, Cohen kappa=0.525
## Per-Class Performance (12 Classification Heads)
### Off-the-Shelf (CardiffNLP Twitter-RoBERTa, ~125M params each)
| Head | Model ID | Classes | Training Data |
|------|----------|---------|--------------|
| Sentiment | cardiffnlp/twitter-roberta-base-sentiment-latest | negative, neutral, positive | TweetEval benchmark |
| Emotion | cardiffnlp/twitter-roberta-base-emotion | anger, joy, optimism, sadness | TweetEval |
| Offensive | cardiffnlp/twitter-roberta-base-offensive | not-offensive, offensive | TweetEval |
| Irony | cardiffnlp/twitter-roberta-base-irony | non-irony, irony | TweetEval |
| Hate | cardiffnlp/twitter-roberta-base-hate-multiclass-latest | not-hate, + 6 subtypes | 13 hate-speech datasets |
| Toxicity | s-nlp/roberta_toxicity_classifier | neutral, toxic | 3 Jigsaw competitions (AUC 0.98) |
CardiffNLP models are pre-trained on 124M tweets. See the TweetEval benchmark
(Barbieri et al., 2020) for per-class F1/P/R on the standard evaluation sets.
### Custom-Trained (SetFit, all-mpnet-base-v2 backbone, ~109M params each)
Trained on 4,121 LLM-labeled tweets from 14 accounts (7 Democrat, 7 Republican).
Evaluated on 20% held-out test set.
| Head | F1 | Precision | Recall | Training Examples | Description |
|------|----|-----------|--------|-------------------|-------------|
| Ragebait | 0.800 | 0.82 | 0.78 | 300 (150+150) | Content designed to provoke outrage |
| Tribal signal | 0.825 | 0.84 | 0.81 | 400 (200+200) | Us-vs-them, in-group/out-group framing |
| Performative outrage | 0.850 | 0.87 | 0.83 | 400 (200+200) | Theatrical outrage vs genuine concern |
| Epistemic manipulation | 0.800 | 0.81 | 0.79 | 300 (150+150) | Cherry-picking, straw-manning, false equiv. |
| Engagement bait | 0.800 | 0.83 | 0.77 | 400 (200+200) | Polls, CTAs, rhetorical questions |
| Agency language | 0.838 | 0.85 | 0.83 | 400 (200+200) | Active/agentic (1) vs passive/victimhood (0) |
### Toxicity Index Components
The per-tweet Toxicity Index is computed as:
```
TI = mean(flag_offensive, flag_toxic, flag_negative_sentiment,
flag_anger, flag_irony, flag_ragebait, flag_tribal,
flag_performative)
```
Where each flag is binary (0 or 1) based on the corresponding classifier threshold.
TI_senator = mean(TI) across all tweets in the archive.
## Compulsion Signature Features
| Feature | Coefficient | Description |
|---------|------------|-------------|
| Time-of-day entropy | +1.258 | Shannon entropy of hourly posting distribution (bits) |
| Hawkes n* | +0.922 | Self-excitation branching ratio |
| Burstiness B | +0.837 | Goh-Barabasi inter-event time parameter |
| Night intensity | +0.584 | Share of posts 00:00-05:59 UTC |
| Weekend ratio | +0.204 | Weekend/weekday posting rate ratio |
## Theoretical Framework
Inspired by Recovery Viability Theory (Kepner, White, & O'Neill, 2026):
- Logit-bounded state space for natural [0,1] constraints
- Cusp catastrophe dynamics for sudden behavioral transitions
- Critical slowing down as early warning signals
## Files
- `bayesian_model_results.json` - Fitted model parameters
- `calibrated_model_v2.json` - V2 validation with independent ground truth
- `cohort_v2_results.csv` - 32-account ground truth cohort
- `cohort_signatures.csv` - Ground truth compulsion signatures
- `setfit_*/` - Trained SetFit classifier checkpoints (6 models)
- `xbox/` - Pipeline source code
## Citation
O'Neill, J., Cabanillas, J., Brooks, J., et al. (2026). Detecting Compulsive Social Media Usage
Patterns in US Congressional Accounts: A Bayesian Temporal Phenotyping Approach.
Manuscript in preparation for International Journal of Drug Policy.
## Ethics
This methodology cannot and should not be used for clinical diagnosis.
The Toxicity Index and compulsion probability are research instruments, not clinical assessments.
|