| | --- |
| | base_model: google/gemma-3-4b-it-unsloth-bnb-4bit |
| | tags: |
| | - text-generation-inference |
| | - transformers |
| | - unsloth |
| | - gemma3 |
| | - trl |
| | license: apache-2.0 |
| | language: |
| | - en |
| | --- |
| | |
| | # 🧠 Fine-Tuning Gemma 3B on Healthcare Admin Tasks |
| |
|
| | This repository demonstrates how to fine-tune the instruction-tuned [`google/gemma-3-4b-it`](https://huggingface.co/google/gemma-3-4b-it) model on a custom dataset covering administrative tasks in the healthcare industry. |
| |
|
| | --- |
| |
|
| | ## 📌 Project Overview |
| |
|
| | We use the [Unsloth](https://github.com/unslothai/unsloth) framework to: |
| | - Load and quantize the base Gemma model in **4-bit precision**. |
| | - Apply **LoRA (Low-Rank Adaptation)** for efficient parameter tuning. |
| | - Train the model using Hugging Face's `trl` library and `SFTTrainer`. |
| |
|
| | This setup significantly reduces memory footprint and training cost, making it suitable for training on consumer GPUs (e.g. Colab, T4, A100). |
| |
|
| | --- |
| |
|
| | ## 🩺 Dataset: Healthcare Admin |
| |
|
| | - **Source**: [`xgalaxy/healthcare_admin`](https://huggingface.co/datasets/xgalaxy/healthcare_admin) |
| | - **Format**: ShareGPT-style JSON format with structured `user` and `assistant` roles |
| | - **Coverage**: |
| | - Appointment scheduling, cancellation, and rescheduling |
| | - Edge cases involving follow-ups, missing info, and ambiguous requests |
| | - Multi-turn conversations to emulate real-world interactions |
| |
|
| | --- |
| |
|
| | ## 🛠️ Key Components |
| |
|
| | ### ✅ Model Setup |
| | - `google/gemma-3-4b-it` loaded using Unsloth's `FastModel.from_pretrained()` |
| | - 4-bit quantization enabled via `load_in_4bit=True` |
| | - LoRA adapters injected for memory-efficient tuning |
| |
|
| | ### ✅ Training |
| | - Supervised fine-tuning with `SFTTrainer` |
| | - Batch size simulated using `gradient_accumulation_steps` |
| | - Linear learning rate scheduler with warmup |
| | - Training capped at a fixed number of steps for fast iteration |
| |
|
| | --- |
| |
|
| | ## 🚀 Trained Model |
| |
|
| | The fine-tuned model is available on Hugging Face: |
| | 👉 [xgalaxy/gemma-3](https://huggingface.co/xgalaxy/gemma-3) |
| |
|
| | --- |
| |
|
| | ## 🔗 Resources |
| |
|
| | - 🔗 [Unsloth GitHub](https://github.com/unslothai/unsloth) |
| | - 📘 [Gemma on Hugging Face](https://huggingface.co/google/gemma-3-4b-it) |
| | - 🗃️ [Healthcare Admin Dataset](https://huggingface.co/datasets/xgalaxy/healthcare_admin) |
| |
|
| | --- |
| |
|
| |
|
