| --- |
| base_model: microsoft/Phi-3-mini-4k-instruct |
| library_name: peft |
| pipeline_tag: text-classification |
| tags: |
| - base_model:adapter:microsoft/Phi-3-mini-4k-instruct |
| - lora |
| - transformers |
| - text-classification |
| - data-governance |
| - pii-detection |
| license: mit |
| language: |
| - en |
| --- |
| |
| # phi3-mini-sensitive-lora |
|
|
| A LoRA adapter that fine-tunes **Phi-3-mini** to classify text as **sensitive |
| (1)** or **not sensitive (0)** — for detecting PII, financial data, credentials, |
| HR/medical info, and government IDs mixed into company databases. |
|
|
| Part of the **AI Data Governance Platform** project. |
|
|
| ## Results (held-out test set, n = 900) |
|
|
| | Metric | Baseline (untrained head) | Fine-tuned (this adapter) | |
| |--------|---------------------------|---------------------------| |
| | Accuracy | 0.529 | **1.000** | |
| | Precision | 0.645 | **1.000** | |
| | Recall | 0.091 | **1.000** | |
| | F1 | 0.159 | **1.000** | |
|
|
| *Note: 1.00 reflects strong learning of a controlled synthetic dataset; |
| real-world data would need a human-labeled test set to confirm generalization.* |
|
|
| ## Training |
|
|
| - **Base model:** microsoft/Phi-3-mini-4k-instruct (loaded in 4-bit / QLoRA) |
| - **Method:** LoRA (rank 16, α 32, dropout 0.05) on the sequence-classification head |
| - **Trainable params:** 25.2M (0.67%) |
| - **Data:** 6,000 synthetic supermarket records (Indian locale), 70/15/15 split |
| - **Hardware:** RTX 3060 12GB, ~38 min, 3 epochs |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| from peft import PeftModel |
| from transformers import AutoModelForSequenceClassification, AutoTokenizer, BitsAndBytesConfig |
| |
| BASE = "microsoft/Phi-3-mini-4k-instruct" |
| ADAPTER = "shivam14245/phi3-mini-sensitive-lora" |
| |
| tok = AutoTokenizer.from_pretrained(BASE) |
| tok.pad_token = tok.pad_token or tok.eos_token |
| |
| bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", |
| bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True) |
| model = AutoModelForSequenceClassification.from_pretrained(BASE, num_labels=2, |
| quantization_config=bnb, device_map="cuda") |
| model.config.pad_token_id = tok.pad_token_id |
| model = PeftModel.from_pretrained(model, ADAPTER).eval() |
| |
| text = "employee salary 85000 bank_account 9876543210 ifsc HDFC0001234" |
| inputs = tok(text, return_tensors="pt", truncation=True, max_length=256).to("cuda") |
| label = model(**inputs).logits.argmax(-1).item() # 1 = sensitive, 0 = not |
| print("sensitive" if label == 1 else "not sensitive") |
| ``` |
|
|
| ## Labels |
| `0` = not sensitive · `1` = sensitive (pii / financial / credentials / hr_medical / govt_id) |
|
|
