---
title: Hackthon2
emoji: 🧠
colorFrom: blue
colorTo: purple
sdk: streamlit
app_file: app.py
pinned: false
---
# MindSignal: Mental Health Support Triage Assistant

MindSignal is a hackathon prototype that classifies short mental-health-related messages into three triage labels:

- `informational`
- `emotional_support_needed`
- `escalation_required`

## Problem Statement

Support teams, community moderators, and wellness platforms often receive short messages that vary widely in urgency. MindSignal explores whether a lightweight NLP assistant can help triage these messages into informational requests, emotional support needs, and possible escalation cases.

This project is a prototype only. It is not a medical diagnosis tool, therapist, crisis service, or emergency system.

## Model Used

The project uses the pre-trained Hugging Face model `distilbert-base-uncased`.

DistilBERT is a smaller and faster version of BERT. It already understands a broad amount of English language structure from pre-training, which makes it useful for transfer learning on a small hackathon dataset.

## Transfer Learning Approach

`train.py` loads the pre-trained DistilBERT weights and adds a 3-class sequence classification head:

- class 0: `informational`
- class 1: `emotional_support_needed`
- class 2: `escalation_required`

The model is then fine-tuned on rows where `split == "train"` in:

```text
data/mental_health_triage_synthetic_dataset.csv
```

The fine-tuned model is saved to:

```text
models/mindsignal-distilbert/
```

## Data Augmentation

The training pipeline keeps the original examples and adds lightweight augmented copies using:

- random lowercase conversion
- small typo/noise injection
- emoji insertion
- slang phrase insertion
- abbreviation examples such as `rn`, `ngl`, and `idk`

These augmentations are intentionally simple and are used only on the training split.

## Safety Override

Before model prediction, MindSignal checks for high-risk phrases such as:

- `kill myself`
- `end my life`
- `not worth living`
- `hurt myself`
- `suicide`
- `not safe`
- `better off without me`
- `can't keep myself safe`

If one of these phrases is found, the prediction is immediately set to:

```text
escalation_required
```

This rule-based override is used in both `evaluate.py` and `app.py`.

## Evaluation Metrics

`evaluate.py` evaluates the test split and stress-test split separately.

It reports:

- accuracy
- macro F1
- per-class precision, recall, and F1
- `escalation_required` recall
- confusion matrix
- stress-test accuracy

Outputs are saved to:

```text
results/evaluation_report.txt
results/confusion_matrix.png
```

## Preliminary Results

After training, run evaluation to generate preliminary results:

```bash
python evaluate.py
```

The generated report will contain the current model's metrics on the `test` and `stress_test` splits. Because this repository includes a very small synthetic starter dataset, results are useful only as a smoke test. Replace the CSV with a larger reviewed dataset for meaningful model performance.

## Project Structure

```text
.
+-- app.py
+-- evaluate.py
+-- train.py
+-- mindsignal_utils.py
+-- requirements.txt
+-- README.md
+-- data/
|   +-- mental_health_triage_synthetic_dataset.csv
+-- models/
|   +-- mindsignal-distilbert/
+-- results/
    +-- evaluation_report.txt
    +-- confusion_matrix.png
```

## Setup

Use Python 3.11.

```bash
python3.11 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

## Run Training

```bash
python train.py
```

This fine-tunes DistilBERT and saves the trained model in `models/mindsignal-distilbert/`.

## Run Evaluation

```bash
python evaluate.py
```

This writes `results/evaluation_report.txt` and `results/confusion_matrix.png`.

## Run the Streamlit App

```bash
streamlit run app.py
```

The app provides:

- a text area for a user message
- a classify button
- predicted label
- confidence score
- safety warning when `escalation_required` is predicted
- a clear medical and emergency-service disclaimer

## Limitations

- The included dataset is synthetic and small.
- The classifier should not be used as a standalone safety system.
- High-risk language can be indirect, sarcastic, multilingual, or misspelled.
- False negatives are especially serious for escalation cases.
- Human review and professional crisis workflows are required for real deployments.
- More diverse data, expert labeling, calibration, fairness testing, and red-team evaluation are needed before real-world use.
## Attribution

This project was developed with assistance from OpenAI ChatGPT for software engineering support, code generation, testing, and documentation. All generated content was reviewed, modified, and validated by the author.

Additional references:
- PyTorch Documentation: https://pytorch.org/docs/stable/
- scikit-learn Documentation: https://scikit-learn.org/stable/