---
base_model: meta-llama/Llama-3.2-3B-Instruct
library_name: peft
pipeline_tag: text-generation
license: llama3.2
tags:
  - llama
  - lora
  - peft
  - qlora
  - biomedical
  - healthcare
  - pubmed
  - medical
  - question-answering
---

# HealthGPT-LoRA

HealthGPT-LoRA is a biomedical question-answering model built by fine-tuning **Meta Llama 3.2 3B Instruct** using **QLoRA (PEFT)** on the **PubMedQA** dataset.

This repository contains **only the LoRA adapter**, which can be loaded on top of the original Llama 3.2 3B Instruct model.

---

# Model Details

- **Base Model:** Meta Llama 3.2 3B Instruct
- **Fine-tuning Method:** QLoRA (PEFT)
- **Task:** Biomedical Question Answering
- **Framework:** Transformers + PEFT
- **Quantization:** 4-bit NF4 (BitsAndBytes)
- **Precision:** BF16 Mixed Precision

---

# Dataset

The model is trained on the **PubMedQA** dataset containing biomedical question-answer pairs.

### Current Training Progress

- **Training Samples:** ~150,000
- **Dataset Completion:** Approximately **75%**
- Training is currently in progress, with plans to continue training on the remaining dataset and further improve performance.

---

# Current Evaluation Results

| Metric | Score |
|--------|------:|
| Accuracy | **93.4%** |
| Precision | **98.1%** |
| Recall | **93.4%** |
| F1 Score | **95.7%** |

The current model achieves approximately **12% higher accuracy** than the base Llama 3.2 3B Instruct model on the evaluation dataset.

---

# Planned Improvements

Upcoming milestones include:

- Retrieval-Augmented Generation (RAG)
- FAISS Vector Database
- Multi-source Medical Knowledge Retrieval
- FastAPI Backend
- Docker Deployment
- CI/CD Pipeline
- Web-based Interface

---

# Usage

```python
from transformers import AutoModelForCausalLM
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-3B-Instruct"
)

model = PeftModel.from_pretrained(
    base_model,
    "llmithull/HealthGPT-LoRA"
)
```

---

# Repository Contents

This repository includes:

- LoRA Adapter Weights
- Adapter Configuration
- Training Configuration

The original Llama 3.2 model is **not included** and must be downloaded separately from Hugging Face.

---

# Project Status

🚧 **Active Development**

HealthGPT is an ongoing project focused on building a production-ready biomedical AI assistant. The current release represents approximately **75% of the planned fine-tuning process**, with Retrieval-Augmented Generation (RAG) and deployment planned in future updates.