dss107/LLAMA-3.2-1b-HRV-Insights

🧠 Fine-tuning Llama 3.2-1B-Instruct with LoRA (4-bit Quantization)

📅 Training Summary

Date: 2025-10-13
Framework: Hugging Face Transformers + PEFT + bitsandbytes
Model Base: meta-llama/Llama-3.2-1B-Instruct
Adapter Type: LoRA (QLoRA 4-bit)

---

⚙️ Environment Setup

pip install -q datasets transformers peft accelerate bitsandbytes huggingface_hub

---

🧾 Dataset Format

Training data is provided in a .jsonl file (output.jsonl) where each sample contains conversational messages in chat format:

{
  "messages": [
    {"role": "system", "content": "You are a health insights assistant."},
    {"role": "user", "content": "- RMSSD: 35.4\n- StressScore: 70.1"},
    {"role": "assistant", "content": "Your stress levels are high, suggesting reduced recovery ability."}
  ]
}

---

🧩 Model and Quantization Setup

The base model is loaded in 4-bit precision for memory efficiency using BitsAndBytesConfig:

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

The LoRA configuration modifies attention and projection layers to enable efficient fine-tuning. The Target modules are:

    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"]
)

---

🧠 Training Configuration

Parameter	Value
Batch Size	1
Gradient Accumulation	4
Warmup Steps	50
Max Steps	500
Learning Rate	2e-4
Scheduler	Cosine
Precision	bfloat16
Optimizer	paged_adamw_8bit
Gradient Checkpointing	✅
Total Epochs	10

---

🏋️ Training Results

Final Output:

TrainOutput(global_step=500, training_loss=0.3287, metrics={
    'train_runtime': 8079.93,
    'train_samples_per_second': 0.248,
    'train_steps_per_second': 0.062,
    'total_flos': 1.25e+16,
    'train_loss': 0.3287,
    'epoch': 10.0
})

Model Output: Generating response for new user data...

--- Health Insights Assistant Response --- Parameter Insights: Baseline: Your long-term baseline shows no major changes, indicating stable recovery patterns. MeanRR: Your rhythm pacing is balanced, keeping an optimal heart rhythm for your body. HR: Your heart rate is moderate, which is a healthy level for sustained recovery. SDNN: Your variability reserve is limited, showing a low overall flexibility in your system. RMSSD: Your short-term adaptability is very low, suggesting a limited capacity for immediate recovery. pNN50: Your adaptability consistency is weak, showing very little variability in your recovery response. AMo50: Your rhythm regularity is stable, maintaining an optimal level of coordination. MxDMn: Your variability spread is balanced, keeping an optimal level of dynamic responsiveness. CV: Your proportional variability is low, indicating a limited efficiency in your system’s flexibility. lnRMSSD: Your recovery efficiency is stable, showing an optimal and consistent level of relaxation. StressScore: Your stress levels are moderate, meaning your system is managing but still alert.

Overall Interpretation: Your Apple data shows stable rhythm pacing and recovery, though your variability is limited and stress levels remain moderate. You’re managing well, but could benefit from further variability expansion.

Actionable Suggestions: - Prioritize short breaks during your most mentally demanding tasks. - Add recovery-friendly activities like stretching or meditation to balance your workload. - Support long-term variability improvement with consistent rest and hydration.