CareCounselor-LLaMA-3.1-8B-Instruct

🧠 Model Overview

CareCounselor-LLaMA-3.1-8B-Instruct is a fine-tuned version of Meta’s LLaMA-3.1-8B-Instruct, specialized for student mental health counseling. It is trained to generate empathetic, relevant, and supportive responses in mental health and counseling contexts.

This model aims to assist in:

• Building conversational agents for student mental well-being
• Offering support through educational mental health platforms
• Providing first-level emotional assistance (non-clinical)

---

📚 Dataset

• arafatanam/Student-Mental-Health-Counseling-50K
  A dataset of 50,000 conversational samples related to counseling, emotional support, anxiety, stress, and student life challenges.

---

⚙️ Training Details

• Base Model: meta-llama/Llama-3.1-8B-Instruct
• Fine-Tuning Technique: LoRA using Unsloth
• Hardware: Kaggle Notebook with 2x T4 GPUs
• Tokenizer: Original tokenizer from base model
• Max Sequence Length: 512
• Training Framework: SFTTrainer with Hugging Face transformers

🔧 Training Arguments

• per_device_train_batch_size = 1
• gradient_accumulation_steps = 8
• num_train_epochs = 1
• learning_rate = 2e-4
• optimizer = adamw_8bit
• lr_scheduler_type = cosine
• fp16: enabled
• warmup_ratio = 0.01
• weight_decay = 0.01
• logging_steps = 100
• save_steps = 1000
• seed = 3407

---

📉 Training Performance

Step	Training Loss
100	1.4947
1000	0.9752
2000	0.8895
3000	0.8624
3100	0.8756

• Final Loss: 0.8756
• Total Steps: 3125
• Total FLOPs: 390,573,497,511,198,720
• Train Runtime: ~10.5 hours
• Train Samples/sec: 1.325
• Train Steps/sec: 0.083

---

💡 Use Cases

• AI-driven mental health chatbots
• Student-focused counseling apps
• Educational well-being assistants
• Stress and emotional support tools

---

🚀 How to Use

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "arafatanam/CareCounselor-LLaMA-3.1-8B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

input_text = "I’ve been feeling anxious and overwhelmed with exams. What should I do?"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=150)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))