| | --- |
| | library_name: transformers |
| | tags: |
| | - gemma |
| | - psychology |
| | - mental-health |
| | - lora |
| | - instruction-tuning |
| | - neuroquestai |
| | license: mit |
| | datasets: |
| | - jkhedri/psychology-dataset |
| | language: |
| | - en |
| | metrics: |
| | - perplexity |
| | base_model: |
| | - google/gemma-2b-it |
| | pipeline_tag: text-generation |
| | --- |
| | |
| | # Model Card for Gemma-2b-it-Psych |
| |
|
| | ## Model Summary |
| |
|
| | **Gemma-2b-it-Psych** is a domain-adapted version of `google/gemma-2b-it`, fine-tuned using LoRA on an instruction-based psychology dataset. |
| | The model is optimized to generate **empathetic, supportive, and professionally aligned psychological responses**, primarily for educational and research purposes. |
| |
|
| | This repository contains **LoRA adapters only**. The base model must be loaded separately. |
| |
|
| | --- |
| |
|
| | ## Model Details |
| |
|
| | ### Model Description |
| |
|
| | - **Author:** Ederson Corbari (e@NeuroQuest.ai) |
| | - **Date:** February 01, 2026 |
| | - **Model type:** Causal Language Model (LLM) |
| | - **Language(s):** English |
| | - **License:** Same as base model (`google/gemma-2b-it`) |
| | - **Finetuned from model:** `google/gemma-2b-it` |
| | - **Fine-tuning method:** LoRA / QLoRA (parameter-efficient fine-tuning) |
| |
|
| | This model was fine-tuned using instruction–response pairs focused on psychological support. |
| | Only empathetic and therapeutically appropriate responses were retained during training, while judgmental or aggressive alternatives were excluded. |
| |
|
| | --- |
| |
|
| | ### Model Sources |
| |
|
| | - **Hugging Face Repository:** https://huggingface.co/ecorbari/Gemma-2b-it-Psych |
| | - **GitHub Repository:** https://github.com/edersoncorbari/fine-tune-llm |
| | - **Base Model:** https://huggingface.co/google/gemma-2b-it |
| |
|
| | --- |
| |
|
| | ## Uses |
| |
|
| | ### Direct Use |
| |
|
| | This model is intended for: |
| |
|
| | - Research and experimentation with instruction-tuned LLMs |
| | - Educational demonstrations of LoRA fine-tuning |
| | - Prompt engineering and behavioral analysis in psychology-related domains |
| |
|
| | The model requires the base Gemma-2B weights to be loaded together with the LoRA adapters. |
| |
|
| | --- |
| |
|
| | ### Downstream Use |
| |
|
| | - Further fine-tuning on related domains |
| | - Adapter merging to create a standalone model |
| | - Quantization for efficient local inference (e.g., GGUF formats) |
| |
|
| | --- |
| |
|
| | ### Out-of-Scope Use |
| |
|
| | - Clinical diagnosis or treatment |
| | - Real-world mental health interventions without professional supervision |
| | - High-stakes decision-making |
| | - Autonomous counseling systems |
| |
|
| | --- |
| |
|
| | ## Bias, Risks, and Limitations |
| |
|
| | - The model may generate inaccurate or incomplete information. |
| | - It does not replace licensed mental health professionals. |
| | - Responses may reflect biases present in the training data. |
| | - Empathy does not guarantee correctness or safety in all contexts. |
| |
|
| | --- |
| |
|
| | ### Recommendations |
| |
|
| | Users should apply human oversight, especially in sensitive scenarios. |
| | This model is best suited for **research, learning, and proof-of-concept applications**. |
| |
|
| | --- |
| |
|
| | ## How to Get Started with the Model |
| |
|
| | ```python |
| | import torch |
| | |
| | from transformers import AutoTokenizer, AutoModelForCausalLM |
| | from peft import PeftModel |
| | |
| | base_model = "google/gemma-2b-it" |
| | adapter_model = "ecorbari/Gemma-2b-it-Psych" |
| | |
| | tokenizer = AutoTokenizer.from_pretrained(base_model) |
| | model = AutoModelForCausalLM.from_pretrained( |
| | base_model, dtype=torch.float16, device_map="auto" |
| | ) |
| | |
| | model = PeftModel.from_pretrained(model, adapter_model) |
| | |
| | prompt = "How can I cope with anxiety during stressful situations?" |
| | inputs = tokenizer(prompt, return_tensors="pt").to(model.device) |
| | |
| | outputs = model.generate(**inputs, max_new_tokens=150) |
| | print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| | ``` |
| |
|
| | ## Evaluation |
| |
|
| | ### Testing Data |
| |
|
| | The model was evaluated on a held-out validation split of the psychology instruction dataset used during fine-tuning. |
| |
|
| | --- |
| |
|
| | ### Metrics |
| |
|
| | The following metrics were used to evaluate the model during training: |
| |
|
| | - Cross-entropy loss (per token) |
| | - Perplexity (exp(loss)) |
| |
|
| | --- |
| |
|
| | ### Results |
| |
|
| | | Metric | Value (approx.) | |
| | |------------|-----------------| |
| | | Eval Loss | 0.60 – 0.70 | |
| | | Perplexity | 1.8 – 2.0 | |
| |
|
| | Perplexity was computed as the exponential of the evaluation loss. Lower values indicate higher confidence in next-token prediction. |
| |
|
| | These metrics reflect convergence and generalization within the target domain, but do not directly assess clinical correctness or psychological safety. |
| |
|
