File size: 2,524 Bytes
2c9399c 5e2ca75 2c9399c 5e2ca75 2c9399c 5e2ca75 2c9399c 5e2ca75 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 | ---
license: apache-2.0
base_model: Qwen/Qwen2.5-1.5B-Instruct
tags:
- lora
- peft
- fraud-detection
- text-classification
---
# LoRA Fine-Tuned Qwen2.5-1.5B for Fraud Risk Classification
This model is a LoRA fine-tuned version of [Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct),
adapted to classify KYC/transaction notes as `suspicious` or `not_suspicious`, with a one-line reason —
inspired by real-world fraud detection and identity verification use cases.
## Model Details
- **Base model:** Qwen/Qwen2.5-1.5B-Instruct
- **Fine-tuning method:** LoRA (PEFT), applied to attention layers (`q_proj`, `v_proj`), r=16, alpha=32
- **Quantization:** Loaded in 4-bit (NF4) during training
- **Training data:** ~297 synthetically generated labeled examples covering identity mismatches,
unusual transfer patterns, dormant-account activity, and document tampering, alongside routine
legitimate transactions
- **Trained on:** Free Google Colab T4 GPU, 3 epochs (~96 seconds total)
## Results
| Stage | Accuracy |
|---|---|
| Zero-shot baseline (no fine-tuning) | ~60-70% (biased toward false positives) |
| After LoRA fine-tuning | **100%** (53/53 on held-out test set) |
The base model consistently over-flagged routine transactions (groceries, rent, tax refunds)
as suspicious. Fine-tuning on balanced examples corrected this bias.
## How to Use
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(base_model, "ishank9/lora-fraud-classifier")
tokenizer = AutoTokenizer.from_pretrained("ishank9/lora-fraud-classifier")
prompt = "Classify the following note as 'suspicious' or 'not_suspicious' and give a one-line reason.\n\nNote: A dormant account suddenly received ₹900,000 and transferred it out the same day.\n\nAnswer:"
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```
## Limitations
This model was trained on a relatively small, LLM-generated synthetic dataset. While it performs
strongly on similarly-styled held-out examples, this reflects pattern-learning on clean synthetic
data rather than a guarantee of real-world generalization to messier, real transaction data.
## Links
- GitHub repo (training notebook, dataset, full write-up): https://github.com/IshankAggarwal09/lora-fraud-classifier |