Language-based assessment (LBA) model for depression
Purpose: Predicts depressive symptom severity from text embeddings using Partial Least Squares regression.
Input: Sentence embeddings generated with
mixedbread-ai/deepset-mxbai-embed-de-large-v1(bilingual DE/EN)Output: Predicted depression symptom score (0-63) on Beck's Depression Inventory, 2nd revision (BDI-II)
LBA model details
Data-driven LBA model. Partial Least Square (PLS) regression with 1 component trained to predict depression scores from sentence embeddings
Predictors: 500 depression symptom narratives (dataset column: "Deptext") described in Gu et al. (2025) and provided with the "topics" package in R (https://github.com/cran/topics/blob/master/data/dep_wor_data.rda)
Criterion: Patient Health Questionnaire 9 item version (PHQ-9) scores, provided in (https://github.com/cran/topics/blob/master/data/dep_wor_data.rda)
Model evaluation using 5-fold cross validation
Final model trained on the entire dataset
Validity evidence
Initial model validation described in Plank et al. (in prep)
Relation with other variables
Covergent evidence: Significant Pearson correlations with BDI-II depression scores across three datasets: Reddit posts (r = 0.39), mental health anamneses (r = 0.14), and depression symptom narratives (r = 0.68)
Divergent evidence: Significantly higher Pearson correlation with depression scores than with anxiety scores (p <= .003) or stress scores (p = .009)
External variables: Significant Pearson correlation with the number of healthcare visits (r = 0.22) and the number of sick leave days (r = 0.18) in the last year in a community sample. No significant point-biserial correlation with primary depression diagnosis made by trained clinical psychologists in a outpatient psychotherapy sample (r = 0.07). No significant point-biserial correlation with current work ability status (able to work vs. on sick leave) in an outpatient psychotherapy sample prior to the beginning of psychotherapeutic treatment (r = 0.02)
Test content
Detection of manual annotations of BDI-II symptom relevance in Reddit sentences. LBA scores could be used to detect symptom relevance above chance for 17/21 BDI-II symptoms. Symptoms Sadness, Irritability, Fatigue, and Loss of interest in sex could not be detected, suggesting slight construct underrepresentation.
Correlation between LBA scores and n-gram frequencies: N-grams correlation positively (e.g., "positive", "optimistic", "relaxed") and negatively (e.g., "bed", "low", "hard") with LBA scores were mostly construct-relevant.
Semantic Projection: Words with positive (e.g., "anguish", "painful", "burden") and negative (e.g., "cheerful", "smoothly", "perfectly") semantic projection scores were mostly construct-relevant.
Response processes
- Representational Construct Similarity: The vector of regression coefficients extracted from the PLS model had a nominally elevated cosine similarity (cos = 0.66) with a semantic vector representation of the BDI-II (anchored contextualized construct representation)
Internal structure
No evidence available yet.
Consequences of testing
No evidence available yet.
Reliability
No evidence available yet.
References
Gu, Z., Kjell, K., Schwartz, H. A., & Kjell, O. (2026). Natural language response formats for assessing depression and worry with large language models: A sequential evaluation with model pre-registration. Assessment, 33(6), 927-953.
Plank, L., Zlomuzica, A., Kjell, O., & Bonnin, G. (in prep). On the validity of psychological assessment with large language models.
Usage
import joblib
from huggingface_hub import hf_hub_download
from sentence_transformers import SentenceTransformer
# Load the bilingual sentence embedding model
sentmod = SentenceTransformer("mixedbread-ai/deepset-mxbai-embed-de-large-v1")
# Load PLS model from Hugging Face Hub
model_path = hf_hub_download(
repo_id="laurinplank/LBA-Depression-Narratives-PLS",
filename="deptextsbdi_PLS.pkl"
)
pls_model = joblib.load(model_path)
text = ["I can barely get out of bed and nothing brings me joy anymore."]
embedding = sentmod.encode(text, show_progress_bar=False)
score = pls_model.predict(embedding)
print(f"Predicted BDI-II score: {score[0]:.2f}")