elchief84 commited on
Commit
d692808
·
verified ·
1 Parent(s): 270a3c1

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +149 -0
README.md ADDED
@@ -0,0 +1,149 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ - it
5
+ license: apache-2.0
6
+ tags:
7
+ - communication
8
+ - tone-detection
9
+ - sentiment-analysis
10
+ - text-classification
11
+ - privacy
12
+ datasets:
13
+ - elchief84/keep-calm-dataset
14
+ metrics:
15
+ - accuracy
16
+ - f1
17
+ - mae
18
+ - pearson-r
19
+ model-index:
20
+ - name: keep-calm
21
+ results:
22
+ - task:
23
+ type: text-classification
24
+ dataset:
25
+ name: Keep Calm test set
26
+ type: workplace-chat
27
+ metrics:
28
+ - name: Risk MAE
29
+ type: mae
30
+ value: 0.100
31
+ - name: Risk Pearson r
32
+ type: pearson-r
33
+ value: 0.700
34
+ - name: Risk Level Accuracy
35
+ type: accuracy
36
+ value: 0.906
37
+ - task:
38
+ type: multi-label-classification
39
+ dataset:
40
+ name: Keep Calm test set
41
+ type: workplace-chat
42
+ metrics:
43
+ - name: Tone Macro F1
44
+ type: f1
45
+ value: 0.677
46
+ - task:
47
+ type: text-classification
48
+ dataset:
49
+ name: Keep Calm test set
50
+ type: workplace-chat
51
+ metrics:
52
+ - name: Intent Accuracy
53
+ type: accuracy
54
+ value: 0.706
55
+ ---
56
+
57
+ # Keep Calm — Communication Risk Analyzer
58
+
59
+ A privacy-first, on-device model for pre-send communication analysis.
60
+ Detects tone, intent, and communication risk in English and Italian text.
61
+
62
+ ## Model description
63
+
64
+ Three independent single-task models sharing a `distilbert-base-multilingual-cased` backbone:
65
+
66
+ - **Risk model**: regression head predicting continuous 0–1 communication risk
67
+ - **Tone model**: multi-label classification across 5 tones
68
+ - **Intent model**: multi-class classification across 4 intents
69
+
70
+ ### Tone labels
71
+
72
+ `neutral` · `frustrated` · `hostile` · `sarcastic` · `positive`
73
+
74
+ ### Intent labels
75
+
76
+ `constructive` · `critical` · `personal` · `informational`
77
+
78
+ ## Intended use
79
+
80
+ Pre-send analysis of workplace text communication. The user writes a message, invokes Keep Calm, sees the analysis, and decides whether to send, revise, or discard.
81
+
82
+ **Not** intended as a moderation or censorship tool. The model estimates perception, not objective truth.
83
+
84
+ ## Out-of-scope use
85
+
86
+ - Automated content moderation
87
+ - Post-hoc message flagging
88
+ - Surveillance or monitoring without consent
89
+ - Analyzing messages in domains other than workplace chat
90
+ - Languages other than English and Italian
91
+
92
+ ## Training data
93
+
94
+ 13,329 annotated examples (English + Italian), workplace chat domain. Sources: YouTube comments, GitHub PRs/issues, LLM-synthesized samples. All labeled by 3+ culturally diverse annotators.
95
+
96
+ ## Bias, risks, and limitations
97
+
98
+ - **Direct communication penalty**: users from direct-communication cultures (German, Dutch) may receive higher risk scores
99
+ - **Sarcasm is hard**: the model's weakest tone (F1 = 0.515); low-confidence predictions are surfaced
100
+ - **Single domain**: trained only on workplace chat; cross-domain performance unmeasured
101
+ - **Context-blind**: no conversation history, relationship context, or cultural cues
102
+ - **Subjective ground truth**: annotator agreement reflects the inherent subjectivity of communication perception
103
+ - **Intent classification**: the weakest task at 70.6% accuracy
104
+
105
+ ## Evaluation results
106
+
107
+ | Task | Metric | Score |
108
+ |---|---|---|
109
+ | Risk | MAE | 0.100 |
110
+ | Risk | Pearson r | 0.700 |
111
+ | Risk | Level accuracy | **90.6%** |
112
+ | Tone | Macro F1 | **0.677** |
113
+ | Intent | Accuracy | **70.6%** |
114
+
115
+ **Latency**: 12.3ms per message on Apple M1 (CPU-only).
116
+
117
+ **Bias audit FP rate**: 15.7% (51 curated probes across 9 categories).
118
+
119
+ ## Hardware
120
+
121
+ - **Inference**: CPU-only, ~400MB RAM, ~12ms per message
122
+ - **Training**: single 16GB GPU (reference: RTX 5060 Ti)
123
+
124
+ ## How to use
125
+
126
+ ```python
127
+ from keep_calm import KeepCalmAnalyzer
128
+
129
+ analyzer = KeepCalmAnalyzer()
130
+ result = analyzer.analyze("Your message here")
131
+
132
+ print(result.communication_risk) # 0.72
133
+ print(result.risk_level) # RiskLevel.HIGH
134
+ print(result.explanation) # human-readable
135
+ ```
136
+
137
+ ## Citation
138
+
139
+ ```bibtex
140
+ @software{keep_calm,
141
+ title = {Keep Calm: Pre-send Communication Risk Analysis},
142
+ year = {2026},
143
+ url = {https://github.com/elchief84/keep-calm}
144
+ }
145
+ ```
146
+
147
+ ## License
148
+
149
+ Apache 2.0