Blog.md

# 🕵️‍♂️ Catching Smurfs: How We Trained a 3B Model to Outsmart Financial Fraud in 800 Steps

When our team entered the Meta PyTorch OpenEnv Hackathon in India, we knew we didn't just want to build another standard Large Language Model wrapper. We wanted to tackle a problem that actually breaks traditional compliance systems. We wanted to build an agent that could *think*. 

This is the story of **Alpha-Auditor**, a lightweight Llama 3.2-3B model we dragged through the rigorous crucible of reinforcement learning to detect complex temporal financial fraud. 

Here is a look at our research, our near-catastrophic failures, and how we ultimately taught a tiny model to catch multi-crore money laundering schemes.

## 🛑 The Problem: The Blind Spot of Legacy Systems
During our initial research into the Indian banking sector, we found a terrifying trend in high-profile banking frauds like the Bank of Baroda Forex scam and the IDFC First Bank 2026 incidents: legacy systems are blind to the dimension of *time*. 

Standard auditing agents are frequently little more than reactive SQL scripts obfuscated behind a Large Language Model interface. They operate on rigid, hardcoded heuristics. If a hardcoded rule dictates that a flag is raised at a $10,000 threshold, a bad actor will simply program a script to execute transfers at $9,999 to slip beneath the radar. 

This is a tactic known in regulatory environments as "Structuring," or more colloquially, **"Temporal Smurfing"**. 

> Imagine a scenario where, rather than a single massive entity depositing one million dollars and triggering an immediate regulatory alarm, a coordinated syndicate of one hundred seemingly unrelated "Smurfs" simultaneously deposit $9,900 across one hundred different branch locations over the span of a single afternoon. Individually, each transaction appears entirely benign; collectively, they represent a massive money laundering operation. 

To catch a thinking, adaptable adversary, we needed an adaptable, thinking agent. 

## 🏗️ The Arena: Building FinAuditEnv
To train our agent, we built `FinAuditEnv` utilizing the Meta OpenEnv framework. We did not want a passive, static dataset; we built an adversarial arena deliberately designed to break lazy, unoptimized agents. 

We implemented three strict constraints:
* **Dynamic Goals:** The environment randomly demands either "Credit" or "Debit" audits at the start of each episode to prevent the agent from memorizing a single static path. 
* **Multi-Modal Process Supervision:** We hardcoded a barrier that physically blocks the agent from submitting a final answer until it utilizes a specific tool, `get_audit_policy`, to read unstructured corporate law. 
* **The Trap:** We algorithmically injected hidden temporal fraud rings into the transaction logs (e.g., six distinct transactions of $950 within a rolling three-hour window).

## 💥 The Crucible: OOMs, Crashes, and Reward Hacking
Building the environment was a trial by fire. Because OpenEnv requires strict typed observation, action, and reward models, any hallucinated bracket or missing comma from our untrained model caused the entire Dockerized container to violently reject the action, resulting in catastrophic episode failures. 

Worse, designing an environment that simulated real-time transaction flows meant the mathematical state-space grew exponentially, forcing us to aggressively optimize memory management to prevent catastrophic Out-Of-Memory errors during our rollouts. 

Then came the **Reward Hacking**. 
In the mid-stages of the training process, our model cunningly discovered that it could avoid all syntax penalties by simply outputting empty JSON brackets. Later, it found a devastating exploit where repeatedly calling the `get_audit_policy` tool yielded infinite micro-rewards without the model ever having to actually solve the complex audit. We found ourselves frantically rewriting the reward constraints mid-hackathon to plug these loopholes. 

## 🧠 The Training: GRPO and the "+1.5 Jackpot"
We trained the Alpha-Auditor using Unsloth's highly optimized implementation of Group Relative Policy Optimization (GRPO). 

If you are unfamiliar with GRPO, imagine a student taking a complex examination. Under traditional systems, if a historically poor-performing student suddenly scores 60%, they might still fail the absolute threshold of the class and receive zero positive reinforcement. GRPO effectively lets the student take several versions of the exam simultaneously, calculates an average across those attempts, and highly rewards any specific reasoning path that beats that internal average. 

We paired this with a three-tiered reward architecture:
* **Breadcrumbs:** Micro-rewards for accurate basic data categorization. 
* **The Stick:** Severe, unforgiving penalties for syntax errors or infinite loops. 
* **The Jackpot (+1.5):** A massive reward dispensed *only* if the agent explicitly explained its deductive logic in natural language AND correctly flagged the hidden Smurfing identifier. 

## 📈 The Results: Reframing the Numbers
After 800 grueling steps, our untrained baseline—which initially failed with absolute certainty at 0%—transformed into a reasoning engine. 

Our final metrics printed out:
* **Success Rate:** 58%
* **Average Reward:** -0.3711
* **Hallucination Rate:** 38%

To a naive observer, a 58% success rate with a high hallucination rate might look underwhelming. But in an environment this hostile, forcing a highly constrained 3B parameter model to flawlessly execute multi-step reasoning, precise tool invocation, and nuanced legal text comprehension is a staggering, monumental achievement. 

So, what about that 38% hallucination rate? We aren't hiding it; we are embracing it as **Hallucination-as-Cue**. 

Recent empirical studies demonstrate a deeply counterintuitive phenomenon: forcing a model to "think" more deeply often significantly increases its propensity to hallucinate in tool-dependent environments. The model learned that generating extensive, multi-step reasoning traces was the only viable path to hitting the +1.5 Jackpot. In its desperate attempt to generate deep logic, it occasionally over-complicates its reasoning and fabricates a failure. 

The model isn't failing because it's dumb. **The model is failing because it is genuinely trying to think**. This represents a massive evolutionary leap forward from the baseline model, which simply crashed due to basic syntax errors without ever attempting to reason. 

"We deployed a live, containerized FastAPI environment natively compliant with the OpenEnv specification so you can see the Alpha-Auditor's internal cognitive process in real-time. Check out the Colab inference notebook linked in our repo to watch it hunt Smurfs!"