| --- |
| base_model: meta-llama/Llama-3.2-3B-Instruct |
| library_name: peft |
| pipeline_tag: text-generation |
| license: llama3.2 |
| tags: |
| - llama |
| - lora |
| - peft |
| - qlora |
| - biomedical |
| - healthcare |
| - pubmed |
| - medical |
| - question-answering |
| --- |
| |
| # HealthGPT-LoRA |
|
|
| HealthGPT-LoRA is a biomedical question-answering model built by fine-tuning **Meta Llama 3.2 3B Instruct** using **QLoRA (PEFT)** on the **PubMedQA** dataset. |
|
|
| This repository contains **only the LoRA adapter**, which can be loaded on top of the original Llama 3.2 3B Instruct model. |
|
|
| --- |
|
|
| # Model Details |
|
|
| - **Base Model:** Meta Llama 3.2 3B Instruct |
| - **Fine-tuning Method:** QLoRA (PEFT) |
| - **Task:** Biomedical Question Answering |
| - **Framework:** Transformers + PEFT |
| - **Quantization:** 4-bit NF4 (BitsAndBytes) |
| - **Precision:** BF16 Mixed Precision |
|
|
| --- |
|
|
| # Dataset |
|
|
| The model is trained on the **PubMedQA** dataset containing biomedical question-answer pairs. |
|
|
| ### Current Training Progress |
|
|
| - **Training Samples:** ~150,000 |
| - **Dataset Completion:** Approximately **75%** |
| - Training is currently in progress, with plans to continue training on the remaining dataset and further improve performance. |
|
|
| --- |
|
|
| # Current Evaluation Results |
|
|
| | Metric | Score | |
| |--------|------:| |
| | Accuracy | **93.4%** | |
| | Precision | **98.1%** | |
| | Recall | **93.4%** | |
| | F1 Score | **95.7%** | |
|
|
| The current model achieves approximately **12% higher accuracy** than the base Llama 3.2 3B Instruct model on the evaluation dataset. |
|
|
| --- |
|
|
| # Planned Improvements |
|
|
| Upcoming milestones include: |
|
|
| - Retrieval-Augmented Generation (RAG) |
| - FAISS Vector Database |
| - Multi-source Medical Knowledge Retrieval |
| - FastAPI Backend |
| - Docker Deployment |
| - CI/CD Pipeline |
| - Web-based Interface |
|
|
| --- |
|
|
| # Usage |
|
|
| ```python |
| from transformers import AutoModelForCausalLM |
| from peft import PeftModel |
| |
| base_model = AutoModelForCausalLM.from_pretrained( |
| "meta-llama/Llama-3.2-3B-Instruct" |
| ) |
| |
| model = PeftModel.from_pretrained( |
| base_model, |
| "llmithull/HealthGPT-LoRA" |
| ) |
| ``` |
|
|
| --- |
|
|
| # Repository Contents |
|
|
| This repository includes: |
|
|
| - LoRA Adapter Weights |
| - Adapter Configuration |
| - Training Configuration |
|
|
| The original Llama 3.2 model is **not included** and must be downloaded separately from Hugging Face. |
|
|
| --- |
|
|
| # Project Status |
|
|
| 🚧 **Active Development** |
|
|
| HealthGPT is an ongoing project focused on building a production-ready biomedical AI assistant. The current release represents approximately **75% of the planned fine-tuning process**, with Retrieval-Augmented Generation (RAG) and deployment planned in future updates. |
