|
|
--- |
|
|
base_model: meta-llama/Llama-3.2-3B-Instruct |
|
|
library_name: peft |
|
|
pipeline_tag: text-generation |
|
|
tags: |
|
|
- base_model:adapter:meta-llama/Llama-3.2-3B-Instruct |
|
|
- lora |
|
|
- transformers |
|
|
- merged_model:ShivomH/Elixir-MentalHealth-3B |
|
|
- mentalhealth |
|
|
- depression |
|
|
- counselling |
|
|
- empathy |
|
|
license: llama3.2 |
|
|
datasets: |
|
|
- ShivomH/MentalHealth-Support |
|
|
language: |
|
|
- en |
|
|
--- |
|
|
|
|
|
# Model Details |
|
|
|
|
|
This is just a LoRA Adapter, please navigate to [ShivomH/Elixir-MentalHealth-3B](https://huggingface.co/ShivomH/Elixir-MentalHealth-3B) to access the merged model with a guided inference script. |
|
|
|
|
|
### Model Description |
|
|
|
|
|
Elixir-MentalHealth is a fine-tuned version of Meta-Llama-3.2-3B-Instruct, adapted using QLoRA on a curated dataset of single-turn and multi-turn mental health support conversations. |
|
|
The model is designed to provide empathetic, safe, and supportive responses while maintaining clear professional boundaries. |
|
|
|
|
|
**⚠️ Disclaimer: This model is not a replacement for professional mental health services. Always seek help from licensed professionals in crisis situations.** |
|
|
|
|
|
### Primary Use Cases: |
|
|
|
|
|
* Mental health support chats |
|
|
* Stress and Anxiety management conversations |
|
|
* Empathetic listening, encouragement and general guidance |
|
|
* Psychoeducational tips (e.g., mindfulness, coping strategies, depression support) |
|
|
|
|
|
### Out-of-Scope Use (should NOT be used for): |
|
|
|
|
|
* Medical diagnosis or treatment planning |
|
|
* Emergency mental health intervention (e.g., suicide prevention crisis line replacement) |
|
|
* Legal, financial, or unrelated domains |
|
|
|
|
|
This model is best suited for research, prototyping, and supportive chatbot applications where professional disclaimers and human oversight are always present. |
|
|
|
|
|
--- |
|
|
|
|
|
## How to Get Started with the Model |
|
|
|
|
|
```Python |
|
|
# Load model with LoRA |
|
|
from peft import PeftModel, PeftConfig |
|
|
|
|
|
lora_model = "ShivomH/Elixir-MentalHealth-3B" |
|
|
base_model = "meta-llama/Llama-3.2-3B-Instruct" |
|
|
|
|
|
# Load configuration |
|
|
peft_config = PeftConfig.from_pretrained(lora_model) |
|
|
|
|
|
# Load base model |
|
|
inference_model = AutoModelForCausalLM.from_pretrained( |
|
|
peft_config.base_model, |
|
|
quantization_config=bnb_config, |
|
|
device_map="auto", |
|
|
torch_dtype=torch.bfloat16, |
|
|
) |
|
|
|
|
|
# Load LoRA weights |
|
|
inference_model = PeftModel.from_pretrained(inference_model, lora_model) |
|
|
|
|
|
# Load tokenizer |
|
|
inference_tokenizer = AutoTokenizer.from_pretrained(lora_model) |
|
|
``` |
|
|
--- |
|
|
|
|
|
# 📊 Dataset Details |
|
|
|
|
|
- **Dataset Source**: [ShivomH/MentalHealth-Support](https://huggingface.co/datasets/ShivomH/MentalHealth-Support) |
|
|
- **Size**: 25,000 conversations |
|
|
- **Training Split**: 23,750 (95%) |
|
|
- **Validation Split**: 1,250 (5%) |
|
|
- **Multi-Turn Conversations**: 16,000 |
|
|
- **Long Single-Turn Conversations**: 8,000 |
|
|
- **Short Single-Turn Conversations**: 1,000 |
|
|
- **Total tokens**: ~17M |
|
|
- **Mean**: ~700 tokens |
|
|
- **Data format**: (.jsonl) Messages List with Roles and Content |
|
|
|
|
|
--- |
|
|
|
|
|
# General Details |
|
|
|
|
|
- **Developed by:** Shivom Hatalkar |
|
|
- **Funded by:** Shivom Hatalkar |
|
|
- **Model type:** NLP Text Generation LLM |
|
|
- **Language(s) (NLP):** English |
|
|
- **License:** llama3.2 |
|
|
- **Base Model:** [meta-llama/Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct) |
|
|
|
|
|
### Model Sources |
|
|
|
|
|
<!-- Provide the basic links for the model. --> |
|
|
|
|
|
- **Repository:** [More Information Needed] |
|
|
- **Paper [optional]:** [More Information Needed] |
|
|
- **Demo [optional]:** [More Information Needed] |
|
|
|
|
|
|
|
|
## Training Details |
|
|
|
|
|
Please visit the Merged model [ShivomH/Elixir-MentalHealth-3B](https://huggingface.co/ShivomH/Elixir-MentalHealth-3B) page for detailed Training details. |
|
|
|
|
|
|
|
|
### Results |
|
|
|
|
|
Please visit the Merged model [ShivomH/Elixir-MentalHealth-3B](https://huggingface.co/ShivomH/Elixir-MentalHealth-3B) page for viewing the testing samples. |
|
|
|
|
|
## Model Examination [optional] |
|
|
|
|
|
<!-- Relevant interpretability work for the model goes here --> |
|
|
|
|
|
[More Information Needed] |
|
|
|
|
|
#### Hardware |
|
|
|
|
|
[More Information Needed] |
|
|
|
|
|
#### Software |
|
|
|
|
|
[More Information Needed] |
|
|
|
|
|
## Citation [optional] |
|
|
|
|
|
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. --> |
|
|
|
|
|
**BibTeX:** |
|
|
|
|
|
[More Information Needed] |
|
|
|
|
|
**APA:** |
|
|
|
|
|
[More Information Needed] |
|
|
|
|
|
### Framework versions |
|
|
|
|
|
- PEFT 0.17.1 |
