pregnancy-qa-qlora

Fine-tuned LoRA adapter for pregnancy domain, optimized for qa tasks.

Version: v2

Model Details

• Base Model: meta-llama/Llama-3.2-3B-Instruct
• Method: QLoRA
• Quantization: 8bit
• LoRA Rank (r): 16
• LoRA Alpha: 32

Evaluation Metrics

• rouge1: 0.4368
• rouge2: 0.2375
• rougeL: 0.3860
• bertscore_precision: 0.9127
• bertscore_recall: 0.9041
• bertscore_f1: 0.9081
• token_f1: 0.3778
• combined_score: 0.5983

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-3B-Instruct",
    device_map="auto",
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "ahmed113hesham/pregnancy-qa-qlora")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")

# Generate
messages = [{"role": "user", "content": "Your question here"}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=256)
    
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Training Details

• Domain: pregnancy
• Training Data: Synthetic QA pairs generated from domain PDFs
• Epochs: 20
• Learning Rate: 0.0001
• Batch Size: 8

Version History

• v2: Current release (2026-01-12)