README.md

---
license: apache-2.0
datasets:
- arafatanam/Mental-Health-Couseling
- arafatanam/Student-Mental-Health-Counseling-10K
language:
- en
base_model:
- unsloth/Qwen2.5-7B-Instruct-bnb-4bit
tags:
- mental-health
- student-focused
- chatbot
---
# Qwen2.5-7B-Instruct Fine-Tuned for Mental Health Counseling

## Model Overview

This is a fine-tuned version of [`unsloth/Qwen2.5-7B-Instruct-bnb-4bit`](https://huggingface.co/unsloth/Qwen2.5-7B-Instruct-bnb-4bit), optimized for mental health counseling conversations. The model is trained to provide supportive, empathetic, and informative responses for individuals seeking mental health support.

## Dataset

The model is fine-tuned on a combination of:

- **Amod/mental_health_counseling_conversations** (cleaned version: [`arafatanam/Mental-Health-Counseling`](https://huggingface.co/datasets/arafatanam/Mental-Health-Counseling)) - **2752 rows**
- **chillies/student-mental-health-counseling-vn** (translated version: [`arafatanam/Student-Mental-Health-Counseling-10K`](https://huggingface.co/datasets/arafatanam/Student-Mental-Health-Counseling-10K)) - **7500 rows**
- **Total dataset size**: 10,252 rows

## Training Details

- **Hardware**: Kaggle Notebooks (GPU T4 x2)
- **Fine-tuning framework**: `Unsloth` with `LoRA`
- **Training settings**:
  - `max_seq_length = 512`
  - `batch_size = 8`
  - `gradient_accumulation_steps = 4`
  - `num_train_epochs = 2`
  - `learning_rate = 5e-5`
  - `optimizer = adamw_8bit`
  - `lr_scheduler = cosine`

## Training Results

- **Final training loss**: `1.3034`
- **Total steps**: `640`
- **Trainable parameters**: `0.58%` of the model`
- **Validation loss**: `1.245`
- **Evaluation metric** (perplexity): `3.42`

## Usage

This model can be used for:

- Providing mental health counseling support
- Generating empathetic responses to mental health queries
- Assisting in chatbot development for mental health applications