| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen2.5-1.5B-Instruct |
| tags: |
| - lora |
| - peft |
| - fraud-detection |
| - text-classification |
| --- |
| |
| # LoRA Fine-Tuned Qwen2.5-1.5B for Fraud Risk Classification |
|
|
| This model is a LoRA fine-tuned version of [Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct), |
| adapted to classify KYC/transaction notes as `suspicious` or `not_suspicious`, with a one-line reason — |
| inspired by real-world fraud detection and identity verification use cases. |
|
|
| ## Model Details |
| - **Base model:** Qwen/Qwen2.5-1.5B-Instruct |
| - **Fine-tuning method:** LoRA (PEFT), applied to attention layers (`q_proj`, `v_proj`), r=16, alpha=32 |
| - **Quantization:** Loaded in 4-bit (NF4) during training |
| - **Training data:** ~297 synthetically generated labeled examples covering identity mismatches, |
| unusual transfer patterns, dormant-account activity, and document tampering, alongside routine |
| legitimate transactions |
| - **Trained on:** Free Google Colab T4 GPU, 3 epochs (~96 seconds total) |
|
|
| ## Results |
| | Stage | Accuracy | |
| |---|---| |
| | Zero-shot baseline (no fine-tuning) | ~60-70% (biased toward false positives) | |
| | After LoRA fine-tuning | **100%** (53/53 on held-out test set) | |
|
|
| The base model consistently over-flagged routine transactions (groceries, rent, tax refunds) |
| as suspicious. Fine-tuning on balanced examples corrected this bias. |
|
|
| ## How to Use |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| from peft import PeftModel |
| |
| base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct") |
| model = PeftModel.from_pretrained(base_model, "ishank9/lora-fraud-classifier") |
| tokenizer = AutoTokenizer.from_pretrained("ishank9/lora-fraud-classifier") |
| |
| prompt = "Classify the following note as 'suspicious' or 'not_suspicious' and give a one-line reason.\n\nNote: A dormant account suddenly received ₹900,000 and transferred it out the same day.\n\nAnswer:" |
| inputs = tokenizer(prompt, return_tensors="pt") |
| output = model.generate(**inputs, max_new_tokens=50) |
| print(tokenizer.decode(output[0], skip_special_tokens=True)) |
| ``` |
|
|
| ## Limitations |
| This model was trained on a relatively small, LLM-generated synthetic dataset. While it performs |
| strongly on similarly-styled held-out examples, this reflects pattern-learning on clean synthetic |
| data rather than a guarantee of real-world generalization to messier, real transaction data. |
|
|
| ## Links |
| - GitHub repo (training notebook, dataset, full write-up): https://github.com/IshankAggarwal09/lora-fraud-classifier |
