File size: 3,140 Bytes
8821d6a 3a22151 222f718 3a22151 8821d6a 3a22151 222f718 3a22151 222f718 3a22151 222f718 3a22151 222f718 8821d6a 222f718 8821d6a 3a22151 8821d6a 3a22151 8821d6a 222f718 3a22151 222f718 3a22151 222f718 3a22151 222f718 3a22151 222f718 3a22151 0966eb9 3a22151 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 | ---
language:
- de
- sv
- en
- fr
- multilingual
license: mit
library_name: transformers
tags:
- text-classification
- democracy
- political-science
- party-competition
- democratic-rhetoric
- mmBert
datasets:
- custom
metrics:
- f1
- accuracy
- precision
- recall
pipeline_tag: text-classification
base_model: jhu-clsp/mmBERT-base
---
# Democracy Detector — Multilingual Modern Bert - Binary Classifier
## Task
Binary classification of sentences from political party press releases:
- **0 — Not democracy**: Sentence does not contain a democratic appeal.
- **1 — Democracy**: Sentence contains a democratic appeal (any rhetorical invocation of democracy, democratic norms, institutions, or principles).
This is **Stage 1** of a two-stage classification pipeline:
1. **Stage 1 (this model)**: Fast binary detection of democracy-related sentences.
2. **Stage 2 (GPT-based)**: Strategy classification of detected sentences (self-assertion, accusation, counter-claim, agenda-setting).
## Model Details
- **Base model**: `jhu-clsp/mmBERT-base`
- **Fine-tuned on**: ~3654 hand-coded sentences from the [PartyPress](https://doi.org/10.7910/DVN/OINX7Q) dataset
- **Languages**: German, Swedish, English, Danish, Polish and Spanish (multilingual press releases)
- **Max sequence length**: 104 tokens
## Training Configuration
| Parameter | Value |
|---|---|
| Learning rate | 0.0001 |
| Epochs | 3 |
| Batch size | 16 |
| Warmup ratio | 0.1 |
| Weight decay | 0.01 |
| Scheduler | cosine |
| Class weights | True |
| Focal loss | False (gamma=2.0) |
| Precision | fp16 |
## Training Data
| Split | Total | Democracy (1) | Not democracy (0) |
|---|---|---|---|
| Train | 3654 | 1512 | 2142 |
| Val | 731 | 205 | 526 |
| Test | 412 | 169 | 243 |
## Performance (Test Set)
precision recall f1-score support
Not democracy 0.907 0.918 0.912 243
Democracy 0.880 0.864 0.872 169
accuracy 0.896 412
macro avg 0.893 0.891 0.892 412
weighted avg 0.895 0.896 0.895 412
## Usage
```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
repo = "LBenoit/democracy-mmBert"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo)
model.eval()
sentence = "Die AfD gefährdet unsere demokratische Grundordnung."
inputs = tokenizer(sentence, return_tensors="pt", truncation=True, max_length=104)
with torch.no_grad():
logits = model(**inputs).logits
prob = torch.softmax(logits, dim=-1)[0, 1].item()
label = "Democracy" if prob >= threshold else "Not democracy"
print(f"{label} (p={prob:.3f})")
```
## Citation
Part of a PhD dissertation on democratic credibility competition in European party systems.
## Author
Léandre Benoit
|