Update README.md
Browse files
README.md
CHANGED
|
@@ -24,4 +24,112 @@ tags:
|
|
| 24 |
|
| 25 |
This model was developed to support standardized, scalable mental health assessments in both clinical and low-resource settings.
|
| 26 |
|
| 27 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
This model was developed to support standardized, scalable mental health assessments in both clinical and low-resource settings.
|
| 26 |
|
| 27 |
+
---
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
## 🧠 Model Details
|
| 31 |
+
|
| 32 |
+
- **Base model**: `bert-base-german-cased`
|
| 33 |
+
- **Task**: Ordinal regression/classification (scores 0–6)
|
| 34 |
+
- **Language**: German 🇩🇪
|
| 35 |
+
- **Input**: Text (dialogue segment grouped by MADRS topic)
|
| 36 |
+
- **Output**: Predicted score for each MADRS item (rounded integer 0–6)
|
| 37 |
+
- **Training data**: Mix of real and synthetic clinician–patient interviews (MADRS-structured)
|
| 38 |
+
|
| 39 |
+
---
|
| 40 |
+
|
| 41 |
+
## 💡 Intended Use
|
| 42 |
+
|
| 43 |
+
This model is intended for research and development use. It is not a certified medical device. The goal is to:
|
| 44 |
+
- Explore AI-assisted symptom severity assessment
|
| 45 |
+
- Enable structured evaluation of individual MADRS items
|
| 46 |
+
- Support clinicians or researchers working in psychiatry/mental health
|
| 47 |
+
|
| 48 |
+
---
|
| 49 |
+
|
| 50 |
+
## 🚀 How to Use
|
| 51 |
+
|
| 52 |
+
### Load model and tokenizer:
|
| 53 |
+
|
| 54 |
+
```python
|
| 55 |
+
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
| 56 |
+
import torch
|
| 57 |
+
|
| 58 |
+
model_name = "webersamantha/MADRS-BERT"
|
| 59 |
+
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
| 60 |
+
model = AutoModelForSequenceClassification.from_pretrained(model_name)
|
| 61 |
+
model.eval().to("cuda" if torch.cuda.is_available() else "cpu")
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
### 📝 Predict on a full structured interview:
|
| 65 |
+
Assume you have a conversation log like this:
|
| 66 |
+
|
| 67 |
+
```python
|
| 68 |
+
conversation_log = [
|
| 69 |
+
{"Speaker": "Interviewer", "Content": "Wie war Ihr Appetit?", "Topic": "Appetit"},
|
| 70 |
+
{"Speaker": "Patient", "Content": "Ich hatte guten Appetit.", "Topic": "Appetit"},
|
| 71 |
+
{"Speaker": "Interviewer", "Content": "Wie war Ihr Schlaf?", "Topic": "Schlaf"},
|
| 72 |
+
{"Speaker": "Patient", "Content": "Ich konnte gut schlafen.", "Topic": "Schlaf"},
|
| 73 |
+
# etc.
|
| 74 |
+
]
|
| 75 |
+
topics = ["Traurigkeit", "Anspannung", "Schlaf", "Appetit", "Konzentration", "Antriebslosigkeit", "Gefühlslosigkeit", "Gedanken", "Suizid"]
|
| 76 |
+
```
|
| 77 |
+
|
| 78 |
+
Use the prediction function:
|
| 79 |
+
|
| 80 |
+
```python
|
| 81 |
+
def predict_scores_per_topic(conversation_log, topics, tokenizer, model):
|
| 82 |
+
device = model.device
|
| 83 |
+
predictions = {}
|
| 84 |
+
for topic in topics:
|
| 85 |
+
topic_dialogue = "\n".join(
|
| 86 |
+
[f"{entry['Speaker']}: {entry['Content']}" for entry in conversation_log if entry["Topic"] == topic]
|
| 87 |
+
)
|
| 88 |
+
if not topic_dialogue:
|
| 89 |
+
predictions[topic] = None
|
| 90 |
+
continue
|
| 91 |
+
inputs = tokenizer(topic_dialogue, truncation=True, padding="max_length", max_length=512, return_tensors="pt").to(device)
|
| 92 |
+
with torch.no_grad():
|
| 93 |
+
score = torch.round(model(**inputs).logits).clamp(0, 6).item()
|
| 94 |
+
predictions[topic] = score
|
| 95 |
+
return predictions
|
| 96 |
+
```
|
| 97 |
+
|
| 98 |
+
---
|
| 99 |
+
|
| 100 |
+
## 🧹 Preprocessing Custom Data
|
| 101 |
+
|
| 102 |
+
If you want to prepare your own data (e.g., from JSONL with structure: `User ID`, `Speaker`, `Transcription`, `Topic`, `Score`), use the preprocessing below:
|
| 103 |
+
|
| 104 |
+
```python
|
| 105 |
+
from datasets import load_dataset
|
| 106 |
+
|
| 107 |
+
dataset = load_dataset("json", data_files="your_data.jsonl", split="train")
|
| 108 |
+
|
| 109 |
+
def preprocess_function(examples):
|
| 110 |
+
scores = [int(float(output.split(":")[1].strip())) for output in examples['output']]
|
| 111 |
+
topics = [
|
| 112 |
+
input_text.split("\n")[0].replace("Topic: ", "").strip()
|
| 113 |
+
if "Topic:" in input_text else "Unknown"
|
| 114 |
+
for input_text in examples['input']
|
| 115 |
+
]
|
| 116 |
+
encoded = tokenizer(examples['input'], truncation=True, padding="max_length", max_length=512)
|
| 117 |
+
encoded["labels"] = scores
|
| 118 |
+
encoded["Topic"] = topics
|
| 119 |
+
return encoded
|
| 120 |
+
|
| 121 |
+
tokenized_dataset = dataset.map(preprocess_function, batched=True)
|
| 122 |
+
```
|
| 123 |
+
|
| 124 |
+
---
|
| 125 |
+
|
| 126 |
+
## 🙏 Acknowledgements
|
| 127 |
+
|
| 128 |
+
Model trained and released by [Samantha Weber](https://github.com/webersamantha). Research conducted as part of efforts to improve AI-driven mental health tools. Thanks to all clinicians and collaborators who contributed to the annotated MADRS dataset.
|
| 129 |
+
|
| 130 |
+
---
|
| 131 |
+
|
| 132 |
+
## 🧪 Citation
|
| 133 |
+
|
| 134 |
+
If you use this model, please cite:
|
| 135 |
+
> Weber, S. et al. (2025). "Using a Fine-tuned Large Language Model for Symptom-based Depression Evaluation" *Preprint*. https://doi.org/10.21203/rs.3.rs-6555767/v1
|