README.md

---
library_name: transformers
tags:
- gemma
- psychology
- mental-health
- lora
- instruction-tuning
- neuroquestai
license: mit
datasets:
- jkhedri/psychology-dataset
language:
- en
metrics:
- perplexity
base_model:
- google/gemma-2b-it
pipeline_tag: text-generation
---

# Model Card for Gemma-2b-it-Psych

## Model Summary

**Gemma-2b-it-Psych** is a domain-adapted version of `google/gemma-2b-it`, fine-tuned using LoRA on an instruction-based psychology dataset.  
The model is optimized to generate **empathetic, supportive, and professionally aligned psychological responses**, primarily for educational and research purposes.

This repository contains **LoRA adapters only**. The base model must be loaded separately.

---

## Model Details

### Model Description

- **Author:** Ederson Corbari (e@NeuroQuest.ai)
- **Date:** February 01, 2026
- **Model type:** Causal Language Model (LLM)
- **Language(s):** English
- **License:** Same as base model (`google/gemma-2b-it`)
- **Finetuned from model:** `google/gemma-2b-it`
- **Fine-tuning method:** LoRA / QLoRA (parameter-efficient fine-tuning)

This model was fine-tuned using instruction–response pairs focused on psychological support.  
Only empathetic and therapeutically appropriate responses were retained during training, while judgmental or aggressive alternatives were excluded.

---

### Model Sources

- **Hugging Face Repository:** https://huggingface.co/ecorbari/Gemma-2b-it-Psych  
- **GitHub Repository:** https://github.com/edersoncorbari/fine-tune-llm 
- **Base Model:** https://huggingface.co/google/gemma-2b-it  

---

## Uses

### Direct Use

This model is intended for:

- Research and experimentation with instruction-tuned LLMs
- Educational demonstrations of LoRA fine-tuning
- Prompt engineering and behavioral analysis in psychology-related domains

The model requires the base Gemma-2B weights to be loaded together with the LoRA adapters.

---

### Downstream Use

- Further fine-tuning on related domains
- Adapter merging to create a standalone model
- Quantization for efficient local inference (e.g., GGUF formats)

---

### Out-of-Scope Use

- Clinical diagnosis or treatment
- Real-world mental health interventions without professional supervision
- High-stakes decision-making
- Autonomous counseling systems

---

## Bias, Risks, and Limitations

- The model may generate inaccurate or incomplete information.
- It does not replace licensed mental health professionals.
- Responses may reflect biases present in the training data.
- Empathy does not guarantee correctness or safety in all contexts.

---

### Recommendations

Users should apply human oversight, especially in sensitive scenarios.  
This model is best suited for **research, learning, and proof-of-concept applications**.

---

## How to Get Started with the Model

```python
import torch

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model = "google/gemma-2b-it"
adapter_model = "ecorbari/Gemma-2b-it-Psych"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(
    base_model, dtype=torch.float16, device_map="auto"
)

model = PeftModel.from_pretrained(model, adapter_model)

prompt = "How can I cope with anxiety during stressful situations?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=150)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Evaluation

### Testing Data

The model was evaluated on a held-out validation split of the psychology instruction dataset used during fine-tuning.

---

### Metrics

The following metrics were used to evaluate the model during training:

- Cross-entropy loss (per token)
- Perplexity (exp(loss))

---

### Results

| Metric     | Value (approx.) |
|------------|-----------------|
| Eval Loss  | 0.60 – 0.70     |
| Perplexity | 1.8 – 2.0       |

Perplexity was computed as the exponential of the evaluation loss. Lower values indicate higher confidence in next-token prediction.

These metrics reflect convergence and generalization within the target domain, but do not directly assess clinical correctness or psychological safety.