parama123/phi3-mini-qlora-guanaco

Phi-3-mini QLoRA Fine-Tuned (3.8B)

QLoRA fine-tuned on Google Colab (T4 GPU) using free tools. Fits in 16GB VRAM.

Quick Stats

• Base: microsoft/Phi-3-mini-4k-instruct (3.8B)
• Method: 4-bit NF4 + LoRA (r=64) via PEFT/bitsandbytes
• Dataset: guanaco-llama2-1k
• Memory: ~14GB (from ~30GB)
• GitHub: paramashiva123/phi3-mini-qlora-finetune

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
import torch

base = AutoModelForCausalLM.from_pretrained(
    "microsoft/Phi-3-mini-4k-instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)
model = PeftModel.from_pretrained(base, "parama123/phi3-mini-qlora-guanaco")
tokenizer = AutoTokenizer.from_pretrained("parama123/phi3-mini-qlora-guanaco")

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
print(pipe("### Human: Explain QLoRA.\n### Assistant:")[0]["generated_text"])

## Training Notebook
[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1mLBrbGIoGg0HLqQYgADO-_Rr20bN32ry?usp=sharing)