---
license: apache-2.0
base_model: Qwen/Qwen2.5-1.5B-Instruct
tags:
- lora
- peft
- fraud-detection
- text-classification
---

# LoRA Fine-Tuned Qwen2.5-1.5B for Fraud Risk Classification

This model is a LoRA fine-tuned version of [Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct), 
adapted to classify KYC/transaction notes as `suspicious` or `not_suspicious`, with a one-line reason — 
inspired by real-world fraud detection and identity verification use cases.

## Model Details
- **Base model:** Qwen/Qwen2.5-1.5B-Instruct
- **Fine-tuning method:** LoRA (PEFT), applied to attention layers (`q_proj`, `v_proj`), r=16, alpha=32
- **Quantization:** Loaded in 4-bit (NF4) during training
- **Training data:** ~297 synthetically generated labeled examples covering identity mismatches, 
  unusual transfer patterns, dormant-account activity, and document tampering, alongside routine 
  legitimate transactions
- **Trained on:** Free Google Colab T4 GPU, 3 epochs (~96 seconds total)

## Results
| Stage | Accuracy |
|---|---|
| Zero-shot baseline (no fine-tuning) | ~60-70% (biased toward false positives) |
| After LoRA fine-tuning | **100%** (53/53 on held-out test set) |

The base model consistently over-flagged routine transactions (groceries, rent, tax refunds) 
as suspicious. Fine-tuning on balanced examples corrected this bias.

## How to Use

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(base_model, "ishank9/lora-fraud-classifier")
tokenizer = AutoTokenizer.from_pretrained("ishank9/lora-fraud-classifier")

prompt = "Classify the following note as 'suspicious' or 'not_suspicious' and give a one-line reason.\n\nNote: A dormant account suddenly received ₹900,000 and transferred it out the same day.\n\nAnswer:"
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```

## Limitations
This model was trained on a relatively small, LLM-generated synthetic dataset. While it performs 
strongly on similarly-styled held-out examples, this reflects pattern-learning on clean synthetic 
data rather than a guarantee of real-world generalization to messier, real transaction data.

## Links
- GitHub repo (training notebook, dataset, full write-up): https://github.com/IshankAggarwal09/lora-fraud-classifier