README.md

---
library_name: transformers
tags:
- parenting
- empathy
license: apache-2.0
language:
- en
metrics:
- bertscore
- rouge
- bleu
base_model:
- mistralai/Mistral-7B-Instruct-v0.3
pipeline_tag: text-generation
---

# Model Card for Model ID

**PretendParentAI** is a fine-tuned variant of `mistralai/Mistral-7B-Instruct-v0.3`, adapted using **Quantized Low-Rank Adaptation (QLoRA)** on Reddit parenting discussions.  
It produces more *empathetic, warm, and relatable* parenting advice.  


## Model Details

### Model Description

**Goal:** Explore how instruction fine-tuning can enhance warmth, relatability, and storytelling in parenting advice LLMs, while assessing trade-offs with factual precision.

**Action:** Fine-tuned Mistral-7B-Instruct on ~40K curated Reddit parenting Q&A pairs (Alpaca format), using Supervised Fine Tuning (SFT) with Parameter-Efficient Fine Tuning (PEFT) i.e. Quantized Low-Rank Adaptation (QLoRA). Built a full instruction-tuning pipeline including Reddit data curation, efficient training/inference using QLoRA, and LLM-as-a-Judge evaluation across empathy, relatability, and other metrics.

**Result:** Produced highly human-like, narrative responses that excelled in empathy (30% to 70%) and relatability (2% to 98%), though often over-personalized or hallucinated personal anecdotes—yielding key insights into the tension between emotional alignment and factual grounding in instruction tuning when using human-generated data (e.g. from reddit).

- **Developed by:** Prerna Chikersal
- **Model type:** PEFT
- **Language(s) (NLP):** English
- **License:** Apache 2.0
- **Finetuned from model:** Mistral-7B-Instruct-v0.3

### Model Sources

- **Repository:** github.com/prernaa/PretendParentAI (includes sample responses)

## Uses

PretendParentAI was developed for **research and educational purposes** — primarily to explore:
- How fine-tuning on real-world dialogue (Reddit parenting discussions) affects **empathy, tone, and relatability** in LLM responses.  
- The trade-off between **emotional resonance** and **clarity/helpfulness** in aligned models.  
- Methods for enhancing warmth and naturalness in conversational AI through **instruction tuning** and **PEFT (QLoRA)**.

Researchers, educators, and ML practitioners can use this model to:
- Study fine-tuning effects on emotional style and alignment.  
- Prototype empathy-driven LLMs for social or psychological dialogue settings.  

### Direct Use

You can use PretendParentAI to:
- Generate **empathetic, supportive, and warm responses** to parenting-related prompts.  
- Experiment with **style transfer and tone control** in conversational AI.  
- Test LLM evaluation metrics (e.g., LLM-as-a-Judge) for empathy, tone, and clarity.

Example:
```python
prompt = "My toddler cries every night before bed. What should I do?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=250)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

### Out-of-Scope Use

This model is not suitable for:
- Clinical, medical, or therapeutic advice.
- Real-world parenting counseling or behavioral guidance.
- Any deployment scenario involving high-stakes decision-making, mental health support, or childcare recommendations.
- Content moderation, bias-free generation, or factual question answering — the Reddit dataset may contain noisy or biased language.

## Bias, Risks, and Limitations
PretendParentAI can **hallucinate personal details** — such as referring to imaginary “sons,” “daughters,” or “partners” — because it imitates how Reddit users often share personal anecdotes. These outputs should not be interpreted as factual or autobiographical.

The model should **not** be used for real parenting, psychological, or medical guidance. Instead, it serves as a research tool for exploring empathy and tone in language models, and all outputs should be reviewed critically before use.

### Recommendations

- Always pair this adapter with the base model mistralai/Mistral-7B-Instruct-v0.3.
- Use bfloat16 precision and FlashAttention 2 on A100 or H100 GPUs for optimal speed.
- Evaluate generations qualitatively for empathy, clarity, and factual accuracy before any downstream use.
- For production or sensitive domains, fine-tune further using curated, high-quality data or Direct Preference Optimization (DPO) to balance warmth and helpfulness.
- When publishing results, acknowledge that this model was trained on Reddit parenting data, which may reflect informal, subjective, and culturally specific communication styles.


## How to Get Started with the Model

This repository only contains **PEFT adapter weights** — not the full 7B model.  
To use the model, you must load the base Mistral model and apply this adapter.

- **Base model:** `mistralai/Mistral-7B-Instruct-v0.3`  
- **Fine-tuning method:** QLoRA (PEFT)  
- **Training data:** Curated Reddit parenting discussions (r/Parenting, r/Mommit, r/Daddit)  
- **Goal:** Explore how instruction tuning on real-world parenting dialogue affects empathy and warmth in responses.  

### How to Load the Model

```python
## Load the base model
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE_MODEL_ID = "mistralai/Mistral-7B-Instruct-v0.3"

torch.backends.cuda.matmul.allow_tf32 = True
torch.set_float32_matmul_precision("high")

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL_ID)

model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL_ID,
    device_map="auto",
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",  # fastest on A100
    token=HF_TOKEN  # or login via huggingface-cli login
)

model.config.pad_token_id = tokenizer.pad_token_id
model.generation_config.pad_token_id = tokenizer.pad_token_id

## Load the PretendParentAI PEFT Model
from peft import PeftModel
model = PeftModel.from_pretrained(model, "your-username/pretendparentai")

## Inference
prompt = """You’re a supportive parent responding to another parent who is struggling with toddler tantrums."""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=300,
    temperature=0.7,
    top_p=0.9
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
## Training Details

### Training Data

Trained on Reddit data from r/parenting. Contact author for dataset info. It can't be publicly shared. 

### Training Procedure

PEFT with QLoRA (4-bit precision) on A100 Google Collab.

#### Training Hyperparameters

- PretendParentAI was fine-tuned using **Quantized Low-Rank Adaptation (QLoRA)** on the base model `mistralai/Mistral-7B-Instruct-v0.3`.  
The model was trained in **4-bit precision** with **double quantization (NF4)** and **bfloat16 compute**, optimized for VRAM efficiency on T4 and A100 GPUs.
The model was trained on A100.

- **Training method:** QLoRA (Parameter-Efficient Fine-Tuning)  
- **Precision:** 4-bit quantization (NF4) with double quantization, compute in bfloat16  
- **Optimizer:** `paged_adamw_8bit`  
- **Scheduler:** Cosine learning rate decay with 3% warmup  
- **Batching:** Effective batch size of 24 (per_device_train_batch_size=6, gradient_accumulation_steps=4)  
- **Epochs:** 1–2 (best checkpoint after 1 epoch, ~1600 steps)  
- **Dropout:** 0.05 (LoRA)  
- **LoRA rank:** 16 (`r=16`), scaling factor `alpha=64`  
- **Trainable parameters:** ~1.12% of total model parameters  
- **Gradient checkpointing:** Enabled  
- **Attention implementation:** FlashAttention 2  
- **Mixed precision:** bfloat16 mixed precision  
- **Base precision (non-quantized runs):** bfloat16  

## Evaluation
We compute BERTScore, Rouge, and BLEU, as well as carry out LLM-as-a-judge evaluation.

### Testing Data, Factors & Metrics

#### Testing Data

Test dataset: https://github.com/prernaa/PretendParentAI/blob/main/data_to_share/reddit_gpt_test_samples.jsonl

[More Information Needed]

#### Metrics and Results

##### Quantitative Metrics

**BLEU**
| Model | BLEU | P@1 | P@2 | P@3 | P@4 | Length Ratio |
|--------|------|-----|-----|-----|-----|---------------|
| Mistral Instruct v0.3 | 0.00695 | 0.1661 | 0.0140 | 0.0023 | 0.0004 | 1.59 |
| PretendParentAI | 0.00624 | 0.1740 | 0.0169 | 0.0016 | 0.0003 | 1.73 |

**ROUGE**
| Model | ROUGE-1 | ROUGE-2 | ROUGE-L | ROUGE-Lsum |
|--------|----------|----------|-----------|--------------|
| Mistral Instruct v0.3 | 0.1774 | 0.0161 | 0.0977 | 0.1040 |
| PretendParentAI | **0.2068** | **0.0215** | **0.1057** | **0.1059** |

**BERTScore (avg.)**
| Model | Precision | Recall | F1 |
|--------|------------|--------|----|
| Mistral Instruct v0.3 | 0.8334 | 0.8440 | 0.8386 |
| PretendParentAI | 0.8323 | 0.8462 | **0.8391** |

> ROUGE and BERTScore show marginal improvement; BLEU remains low (expected for open-ended tasks).

##### LLM-as-a-Judge Evaluation

Evaluation used **GPT-4o** as an LLM judge to compare responses from:
- **System A:** Mistral Instruct v0.3 (base)
- **System B:** PretendParentAI (fine-tuned)

Each system was scored on:  
*helpfulness, empathy/tone, creativity, clarity, relatability, adoptability, and overall preference.*

|              | winner\_helpfulness | winner\_empathy\_tone | winner\_creativity | winner\_clarity | winner\_relatability | winner\_adoptability | winner\_overall |
|--------------|---------------------|-----------------------|--------------------|-----------------|----------------------|----------------------|-----------------|
| **System A** | **0.90**            | 0.30                  | 0.47               | **0.90**        | 0.02                 | **0.72**             | **0.72**        |
| **System B** | 0.10                | **0.70**              | **0.48**            | 0.10            | **0.98**             | 0.28                 | 0.28            |
| **Tie**      | 0.00                | 0.00                  | 0.05               | 0.00            | 0.00                 | 0.00                 | 0.00            |

**Interpretation**
- **System A (Mistral)** remains more *helpful*, *clear*, and *adoptable*.  
- **System B (PretendParentAI)** excels in *empathy* and *relatability*, and is slightly more *creative*.  
- While System A’s responses are helpful 90% of the time, their adoptability is 72% — showing that *users sometimes prefer System B due to emotional tone and resonance, despite lower clarity*.


#### Summary
- Fine-tuning on Reddit data increases empathy and warmth but decreases clarity and factual helpfulness.
- Reddit’s casual, noisy, non-evidence-based writing style limits instruction-tuning quality.
- More empathetic and relatable advice can increase adoptability even if clarity declines.
- Future approaches should explore preference optimization (e.g., DPO) as a way to increase empathy and relatability in responses while maintaining clarity.

## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

- **Hardware Type:** A100
- **Hours used:** 5
- **Cloud Provider:** Google Collab
- **Compute Region:** USA
- **Carbon Emitted:** [More Information Needed]

## Citation
```
@misc{chikersal2025pretendparentai,
author = {Prerna Chikersal},
title = {PretendParentAI: Instruction Fine-Tuning Mistral-7B on Reddit Parenting Data using QLoRA},
year = {2025},
note = {GitHub repository}
}
```

## Model Card Contact

Prerna Chikersal: pchikersal@gmail.com