README.md

---
license: llama3
language:
- en
library_name: peft
base_model: meta-llama/Meta-Llama-3-8B
pipeline_tag: text-generation
tags:
- peft
- lora
- llama-3
- tabular-data
- adolescent-mental-health
---

<a id="readme-top"></a>

<div align="center">
  <h1>MindScale</h1>
  <p><strong>A Psychometric Foundation Model for<br>Population-Scale Adolescent Mental Health Assessment</strong></p>
  <p>
    <img src="https://img.shields.io/badge/PEFT-0.9.0-3776AB" alt="PEFT 0.9.0">
    <img src="https://img.shields.io/badge/Backbone-LLaMA_3_8B-64748B" alt="Backbone: LLaMA 3-8B">
    <a href="https://huggingface.co/Rango001/Adolescent-mental-health"><img src="https://img.shields.io/badge/Weights-LoRA_adapters-FFD21E?logo=huggingface" alt="LoRA adapter weights"></a>
  </p>
  <p>
    <a href="#overview">Overview</a> &middot;
    <a href="#training-details">Training</a> &middot;
    <a href="#loading-adapters">Loading</a> &middot;
    <a href="#intended-use">Use</a> &middot;
    <a href="#limitations">Limitations</a> &middot;
    <a href="#license">License</a>
  </p>
</div>

<a id="overview"></a>

## 📋 Overview

- **Model**

  MindScale is a language-model-based psychometric foundation model adapted from
  [Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) for
  questionnaire-based adolescent mental health screening and psychiatric symptom
  assessment.

- **Released files**

  The release contains LoRA adapter weights and requires the base model for use.

<p align="right"><a href="#readme-top">⬆️ Back to top</a></p>

<a id="training-details"></a>

## 📊 Training Details

- **Dataset**: Large-scale adolescent questionnaire dataset
- **Scale**: 402,301 students from 377 schools
- **Instruments**: 22 questionnaires comprising 250 items
- **Information domains**: Environmental factors, psychological characteristics, and questionnaire-derived psychiatric symptoms
- **Training method**: LoRA-based parameter-efficient fine-tuning
- **Training stages**: Pretraining, at-risk identification, and psychiatric symptom assessment

- **Data access**: The full participant-level training cohort is not included
  because it contains sensitive information from adolescents.

<p align="right"><a href="#readme-top">⬆️ Back to top</a></p>

<a id="loading-adapters"></a>

## 🤗 Loading the Adapters

- **Adapter directories**: `Pretrain/`, `Screen/`, and `Assessment/`.

- **Stage dependencies**: Screening requires the pretraining and screening
  adapters; assessment requires all three.

- **Before running**: Use the MindScale code directory with its runtime
  dependencies installed. The example uses the code release's helpers to prepare
  special tokens and stage adapters consistently.

```python
from pathlib import Path
from huggingface_hub import snapshot_download
from transformers import AutoTokenizer
from configs import (
    SerializerConfig, TokenizerConfig, TrainConfig_Screen, LoraConfig_Screen,
)
from models.llama_model import load_llama_3_lora

base_model_id = "meta-llama/Meta-Llama-3-8B"
adapter_id = "Rango001/Adolescent-mental-health"

weights_dir = Path(snapshot_download(
    repo_id=adapter_id,
    allow_patterns=["*/adapter_config.json", "*/adapter_model.safetensors"],
))
tokenizer = AutoTokenizer.from_pretrained(base_model_id, use_fast=False)
model, tokenizer = load_llama_3_lora(
    tokenizer, SerializerConfig(), TokenizerConfig(), base_model_id,
    task_type="Screen",
    train_config=TrainConfig_Screen(), lora_config=LoraConfig_Screen(),
    pretrain_state_dict_path=str(weights_dir / "Pretrain"),
    screen_state_dict_path=str(weights_dir / "Screen"),
    test_task=True,
)
model.eval()
```

- **Base model access**: Accept the Meta Llama 3 license on Hugging Face.

- **Runtime**: A CUDA-enabled environment is required. The example uses the
  code release's existing 8-bit loading.

- **Questionnaire inference**: Follow the accompanying code README for input
  preparation and prediction export.

<p align="right"><a href="#readme-top">⬆️ Back to top</a></p>

<a id="intended-use"></a>

## 📝 Intended Use

MindScale is intended for:

- **Population screening**: Research on adolescent mental health assessment.
- **Method development**: Studies of structured questionnaire modelling.
- **Model evaluation**: Evaluation on questionnaire-derived tabular data.

<p align="right"><a href="#readme-top">⬆️ Back to top</a></p>

<a id="limitations"></a>

## ⚠️ Limitations

- **Input scope**: The model was trained on questionnaire-based data and may not
  generalize to other input formats or populations.

- **Class imbalance and bias**: Predictions may be affected by class imbalance
  and cohort-specific biases.

- **Clinical validation**: The model has not been validated for routine clinical
  use.

- **Distribution shifts**: Performance may decline under demographic,
  linguistic, institutional, or other distribution shifts.

<p align="right"><a href="#readme-top">⬆️ Back to top</a></p>

<a id="ethical-considerations"></a>

## 🛡️ Ethical Considerations

- **Research use only**: MindScale must not be used for clinical diagnosis,
  treatment decisions, crisis triage, or autonomous decisions affecting students.

- **Prediction uncertainty**: Outputs may be incorrect or biased, particularly
  for underrepresented groups.

- **Participant protection**: Any use involving participant data requires
  appropriate ethical approval, privacy protection, fairness assessment, and
  qualified human oversight.

<p align="right"><a href="#readme-top">⬆️ Back to top</a></p>

<a id="license"></a>

## 📄 License

Use of this adapter and the Meta-Llama-3-8B backbone is subject to the
[Meta Llama 3 Community License](https://llama.meta.com/llama3/license/) and its
Acceptable Use Policy.

<p align="right"><a href="#readme-top">⬆️ Back to top</a></p>