File size: 4,153 Bytes
d692808 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 | ---
language:
- en
- it
license: apache-2.0
tags:
- communication
- tone-detection
- sentiment-analysis
- text-classification
- privacy
datasets:
- elchief84/keep-calm-dataset
metrics:
- accuracy
- f1
- mae
- pearson-r
model-index:
- name: keep-calm
results:
- task:
type: text-classification
dataset:
name: Keep Calm test set
type: workplace-chat
metrics:
- name: Risk MAE
type: mae
value: 0.100
- name: Risk Pearson r
type: pearson-r
value: 0.700
- name: Risk Level Accuracy
type: accuracy
value: 0.906
- task:
type: multi-label-classification
dataset:
name: Keep Calm test set
type: workplace-chat
metrics:
- name: Tone Macro F1
type: f1
value: 0.677
- task:
type: text-classification
dataset:
name: Keep Calm test set
type: workplace-chat
metrics:
- name: Intent Accuracy
type: accuracy
value: 0.706
---
# Keep Calm — Communication Risk Analyzer
A privacy-first, on-device model for pre-send communication analysis.
Detects tone, intent, and communication risk in English and Italian text.
## Model description
Three independent single-task models sharing a `distilbert-base-multilingual-cased` backbone:
- **Risk model**: regression head predicting continuous 0–1 communication risk
- **Tone model**: multi-label classification across 5 tones
- **Intent model**: multi-class classification across 4 intents
### Tone labels
`neutral` · `frustrated` · `hostile` · `sarcastic` · `positive`
### Intent labels
`constructive` · `critical` · `personal` · `informational`
## Intended use
Pre-send analysis of workplace text communication. The user writes a message, invokes Keep Calm, sees the analysis, and decides whether to send, revise, or discard.
**Not** intended as a moderation or censorship tool. The model estimates perception, not objective truth.
## Out-of-scope use
- Automated content moderation
- Post-hoc message flagging
- Surveillance or monitoring without consent
- Analyzing messages in domains other than workplace chat
- Languages other than English and Italian
## Training data
13,329 annotated examples (English + Italian), workplace chat domain. Sources: YouTube comments, GitHub PRs/issues, LLM-synthesized samples. All labeled by 3+ culturally diverse annotators.
## Bias, risks, and limitations
- **Direct communication penalty**: users from direct-communication cultures (German, Dutch) may receive higher risk scores
- **Sarcasm is hard**: the model's weakest tone (F1 = 0.515); low-confidence predictions are surfaced
- **Single domain**: trained only on workplace chat; cross-domain performance unmeasured
- **Context-blind**: no conversation history, relationship context, or cultural cues
- **Subjective ground truth**: annotator agreement reflects the inherent subjectivity of communication perception
- **Intent classification**: the weakest task at 70.6% accuracy
## Evaluation results
| Task | Metric | Score |
|---|---|---|
| Risk | MAE | 0.100 |
| Risk | Pearson r | 0.700 |
| Risk | Level accuracy | **90.6%** |
| Tone | Macro F1 | **0.677** |
| Intent | Accuracy | **70.6%** |
**Latency**: 12.3ms per message on Apple M1 (CPU-only).
**Bias audit FP rate**: 15.7% (51 curated probes across 9 categories).
## Hardware
- **Inference**: CPU-only, ~400MB RAM, ~12ms per message
- **Training**: single 16GB GPU (reference: RTX 5060 Ti)
## How to use
```python
from keep_calm import KeepCalmAnalyzer
analyzer = KeepCalmAnalyzer()
result = analyzer.analyze("Your message here")
print(result.communication_risk) # 0.72
print(result.risk_level) # RiskLevel.HIGH
print(result.explanation) # human-readable
```
## Citation
```bibtex
@software{keep_calm,
title = {Keep Calm: Pre-send Communication Risk Analysis},
year = {2026},
url = {https://github.com/elchief84/keep-calm}
}
```
## License
Apache 2.0
|