README.md

---
base_model: meta-llama/Llama-3.2-3B-Instruct
library_name: peft
pipeline_tag: text-generation
tags:
- base_model:adapter:meta-llama/Llama-3.2-3B-Instruct
- lora
- transformers
- merged_model:ShivomH/Elixir-MentalHealth-3B
- mentalhealth
- depression
- counselling
- empathy
license: llama3.2
datasets:
- ShivomH/MentalHealth-Support
language:
- en
---

# Model Details

This is just a LoRA Adapter, please navigate to [ShivomH/Elixir-MentalHealth-3B](https://huggingface.co/ShivomH/Elixir-MentalHealth-3B) to access the merged model with a guided inference script.

### Model Description

Elixir-MentalHealth is a fine-tuned version of Meta-Llama-3.2-3B-Instruct, adapted using QLoRA on a curated dataset of single-turn and multi-turn mental health support conversations.
The model is designed to provide empathetic, safe, and supportive responses while maintaining clear professional boundaries.

**⚠️ Disclaimer: This model is not a replacement for professional mental health services. Always seek help from licensed professionals in crisis situations.**

### Primary Use Cases:

* Mental health support chats
* Stress and Anxiety management conversations
* Empathetic listening, encouragement and general guidance
* Psychoeducational tips (e.g., mindfulness, coping strategies, depression support)

### Out-of-Scope Use (should NOT be used for):

* Medical diagnosis or treatment planning
* Emergency mental health intervention (e.g., suicide prevention crisis line replacement)
* Legal, financial, or unrelated domains

This model is best suited for research, prototyping, and supportive chatbot applications where professional disclaimers and human oversight are always present.

---

## How to Get Started with the Model

```Python
# Load model with LoRA
from peft import PeftModel, PeftConfig

lora_model = "ShivomH/Elixir-MentalHealth-3B"
base_model = "meta-llama/Llama-3.2-3B-Instruct"

# Load configuration
peft_config = PeftConfig.from_pretrained(lora_model)

# Load base model
inference_model = AutoModelForCausalLM.from_pretrained(
    peft_config.base_model,
    quantization_config=bnb_config,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

# Load LoRA weights
inference_model = PeftModel.from_pretrained(inference_model, lora_model)

# Load tokenizer
inference_tokenizer = AutoTokenizer.from_pretrained(lora_model)
```
---

# 📊 Dataset Details

- **Dataset Source**: [ShivomH/MentalHealth-Support](https://huggingface.co/datasets/ShivomH/MentalHealth-Support)
- **Size**: 25,000 conversations
- **Training Split**: 23,750 (95%)
- **Validation Split**: 1,250 (5%)
- **Multi-Turn Conversations**: 16,000
- **Long Single-Turn Conversations**: 8,000
- **Short Single-Turn Conversations**: 1,000
- **Total tokens**: ~17M
- **Mean**: ~700 tokens
- **Data format**: (.jsonl) Messages List with Roles and Content

---

# General Details

- **Developed by:** Shivom Hatalkar
- **Funded by:** Shivom Hatalkar
- **Model type:** NLP Text Generation LLM
- **Language(s) (NLP):** English
- **License:** llama3.2
- **Base Model:** [meta-llama/Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct)

### Model Sources

<!-- Provide the basic links for the model. -->

- **Repository:** [More Information Needed]
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed]


## Training Details

Please visit the Merged model [ShivomH/Elixir-MentalHealth-3B](https://huggingface.co/ShivomH/Elixir-MentalHealth-3B) page for detailed Training details.


### Results

Please visit the Merged model [ShivomH/Elixir-MentalHealth-3B](https://huggingface.co/ShivomH/Elixir-MentalHealth-3B) page for viewing the testing samples.

## Model Examination [optional]

<!-- Relevant interpretability work for the model goes here -->

[More Information Needed]

#### Hardware

[More Information Needed]

#### Software

[More Information Needed]

## Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**

[More Information Needed]

**APA:**

[More Information Needed]

### Framework versions

- PEFT 0.17.1