File size: 4,920 Bytes
4eb1829 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 | ---
title: Hackthon2
emoji: 🧠
colorFrom: blue
colorTo: purple
sdk: streamlit
app_file: app.py
pinned: false
---
# MindSignal: Mental Health Support Triage Assistant
MindSignal is a hackathon prototype that classifies short mental-health-related messages into three triage labels:
- `informational`
- `emotional_support_needed`
- `escalation_required`
## Problem Statement
Support teams, community moderators, and wellness platforms often receive short messages that vary widely in urgency. MindSignal explores whether a lightweight NLP assistant can help triage these messages into informational requests, emotional support needs, and possible escalation cases.
This project is a prototype only. It is not a medical diagnosis tool, therapist, crisis service, or emergency system.
## Model Used
The project uses the pre-trained Hugging Face model `distilbert-base-uncased`.
DistilBERT is a smaller and faster version of BERT. It already understands a broad amount of English language structure from pre-training, which makes it useful for transfer learning on a small hackathon dataset.
## Transfer Learning Approach
`train.py` loads the pre-trained DistilBERT weights and adds a 3-class sequence classification head:
- class 0: `informational`
- class 1: `emotional_support_needed`
- class 2: `escalation_required`
The model is then fine-tuned on rows where `split == "train"` in:
```text
data/mental_health_triage_synthetic_dataset.csv
```
The fine-tuned model is saved to:
```text
models/mindsignal-distilbert/
```
## Data Augmentation
The training pipeline keeps the original examples and adds lightweight augmented copies using:
- random lowercase conversion
- small typo/noise injection
- emoji insertion
- slang phrase insertion
- abbreviation examples such as `rn`, `ngl`, and `idk`
These augmentations are intentionally simple and are used only on the training split.
## Safety Override
Before model prediction, MindSignal checks for high-risk phrases such as:
- `kill myself`
- `end my life`
- `not worth living`
- `hurt myself`
- `suicide`
- `not safe`
- `better off without me`
- `can't keep myself safe`
If one of these phrases is found, the prediction is immediately set to:
```text
escalation_required
```
This rule-based override is used in both `evaluate.py` and `app.py`.
## Evaluation Metrics
`evaluate.py` evaluates the test split and stress-test split separately.
It reports:
- accuracy
- macro F1
- per-class precision, recall, and F1
- `escalation_required` recall
- confusion matrix
- stress-test accuracy
Outputs are saved to:
```text
results/evaluation_report.txt
results/confusion_matrix.png
```
## Preliminary Results
After training, run evaluation to generate preliminary results:
```bash
python evaluate.py
```
The generated report will contain the current model's metrics on the `test` and `stress_test` splits. Because this repository includes a very small synthetic starter dataset, results are useful only as a smoke test. Replace the CSV with a larger reviewed dataset for meaningful model performance.
## Project Structure
```text
.
+-- app.py
+-- evaluate.py
+-- train.py
+-- mindsignal_utils.py
+-- requirements.txt
+-- README.md
+-- data/
| +-- mental_health_triage_synthetic_dataset.csv
+-- models/
| +-- mindsignal-distilbert/
+-- results/
+-- evaluation_report.txt
+-- confusion_matrix.png
```
## Setup
Use Python 3.11.
```bash
python3.11 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```
## Run Training
```bash
python train.py
```
This fine-tunes DistilBERT and saves the trained model in `models/mindsignal-distilbert/`.
## Run Evaluation
```bash
python evaluate.py
```
This writes `results/evaluation_report.txt` and `results/confusion_matrix.png`.
## Run the Streamlit App
```bash
streamlit run app.py
```
The app provides:
- a text area for a user message
- a classify button
- predicted label
- confidence score
- safety warning when `escalation_required` is predicted
- a clear medical and emergency-service disclaimer
## Limitations
- The included dataset is synthetic and small.
- The classifier should not be used as a standalone safety system.
- High-risk language can be indirect, sarcastic, multilingual, or misspelled.
- False negatives are especially serious for escalation cases.
- Human review and professional crisis workflows are required for real deployments.
- More diverse data, expert labeling, calibration, fairness testing, and red-team evaluation are needed before real-world use.
## Attribution
This project was developed with assistance from OpenAI ChatGPT for software engineering support, code generation, testing, and documentation. All generated content was reviewed, modified, and validated by the author.
Additional references:
- PyTorch Documentation: https://pytorch.org/docs/stable/
- scikit-learn Documentation: https://scikit-learn.org/stable/
|