Spaces:
Sleeping
Download docs/CLASSIFICATION_PIPELINE.md from Coeinterns/bank-statement-api: direct link, hf CLI and curl.
- Browser
- Download file 6.32 kB
-
https://huggingface.co/spaces/Coeinterns/bank-statement-api/resolve/main/docs/CLASSIFICATION_PIPELINE.md
- Command line
-
hf download hf://spaces/Coeinterns/bank-statement-api/docs/CLASSIFICATION_PIPELINE.md
-
curl -L -o CLASSIFICATION_PIPELINE.md https://huggingface.co/spaces/Coeinterns/bank-statement-api/resolve/main/docs/CLASSIFICATION_PIPELINE.md
BeFree Classification Pipeline
This document describes the three-tier classification architecture, keyword matching criteria, auto-learning mechanics, and audit log structures of the BeFree classification pipeline.
Pipeline Overview
The classification pipeline matches transaction logs into standardized expense categories. It uses a sequential fallback model consisting of three engines. The pipeline evaluates the engines in order and terminates at the first engine that yields a non-None classification result:
1. Engine Tiers
1.1 The Rule Engine
The Rule Engine acts as the first, fast, deterministic tier. It retrieves active categorization rules from PostgreSQL (classification_rules and classification_categories tables).
Match Modes:
- Substring Match: Used for seeded rules (priority $\ge 10$) and manual rules. Matches keywords anywhere inside the description string (e.g.
"woolworths"matches"WOOLWORTHS METRO SYDNEY"). - Word-Boundary Match: Automatically enabled for auto-learned rules (priority $< 10$) or short keywords ($\le 4$ characters) to prevent false matches (e.g., keyword
"bar"matches"BAR DE GUSTATION"but does not match"SUBURBAN").- Word boundary patterns are compiled as:
r'(?:^|\b|(?<=\s))' + re.escape(keyword) + r'(?:\b|(?=\s)|$)' - Enabled automatically by
_needs_word_boundary(keyword)if the keyword is $\le 4$ characters, is in a predefined generic boundary keyword list ("bar","pay","shop","online","visa","atm","cc","net"), or contains non-alphanumeric characters.
- Word boundary patterns are compiled as:
Payment Rail Features:
- OSKO / NPP Transfers: Extracts payer names or intent purposes before cleaning description noise.
- If description indicates OSKO/NPP, it extracts the reference payload using regex patterns.
- Matches the extracted name against
osko_referencetarget rules. - If no rule matches, unclassified OSKO debit transfers default to
"Transfers & Internal Movements"or"External Transfer".
- BPAY Lookup: Matches numeric biller codes against the
bpay_billerstable for instant categorization (e.g. biller code7000maps deterministically to"Water Bill & Utilities").
1.2 The ML Engine
The Machine Learning Engine is an offline classifier utilizing TF-IDF and Logistic Regression.
- Status: Disabled by default in settings (
use_ml=False). - Feature Extraction: Character n-grams (range: 2 to 4 characters) to handle typos, merchant name variations, and locations.
- Model Storage: Loaded dynamically from pickle file
data/models/category_classifier.pkl. - Threshold Gates:
MIN_CONFIDENCE = 0.15: Predictions below 15% fall through to the LLM.MIN_MARGIN = 0.05: Rejects predictions where the probability difference between the top-1 and top-2 predicted categories is less than 5% (ambiguous predictions).
1.3 The LLM Engine (Google Gemini)
Serves as the high-accuracy semantic fallback for transactions that did not match rules or ML models.
- Model:
gemini-2.5-flash - Execution Flow:
- Batching: Transactions are grouped into batches of up to 25 (
_BATCH_SIZE = 25). - Concurrency: Batches are processed concurrently in a
ThreadPoolExecutorwith up to 8 threads (_MAX_WORKERS = 8). - Retries: Incorporates exponential back-off retries to handle rate limits (
HTTP 429quota limits) or transient connection timeouts. - Prompt Structure: Details standard categories, provides transaction examples, describes extraction formats, and demands a structured JSON array return detailing
tx_id,category,third_party_name,confidence, andreasoning.
- Batching: Transactions are grouped into batches of up to 25 (
2. Rule Learner & Promotion
The RuleLearner automatically converts frequent LLM classifications into high-speed database rules to minimize future API costs.
2.1 Suggestion Log Criteria
After LLM classification, candidates are written to learned_rule_logs in a pending state if they meet the following:
- Classification Source must be
llm. - LLM confidence score must be $\ge 0.85$ (
MIN_CONFIDENCE_TO_ACTIVATE). - The keyword must not be in the
GENERIC_STOP_WORDSlist (like"payment","transfer","ref", suburb names, etc.) or duplicate active rules.
2.2 Promotion Thresholds
The scripts/clean_and_promote_rules.py script runs periodically (via cron or admin console triggers) to evaluate pending suggestions and promote them to active rules if they meet either threshold:
Once promoted, a rule record is created in classification_rules and the log status changes to approved. Rejected logs are marked rejected.
3. Classification Logging Schema
Every classification decision is recorded in the audit log file data/logs/classification_latest.jsonl.
JSONL Schema Fields:
{
"tx_id": "tx_1_8f93e2",
"logged_at": "2026-06-28T21:42:03.123456Z",
"date": "2026-05-24",
"description": "WOOLWORTHS SUPERMARKETS SYDNEY",
"amount": -74.32,
"tx_type": "debit",
"category": "Food and Groceries",
"source": "rule",
"confidence": 1.0,
"matched_keyword": "woolworths",
"matched_rule_name": "rule_food_groceries",
"ml_top_scores": null,
"llm_prompt_tokens": null,
"llm_response_raw": null,
"rule_learned": false,
"learned_keyword": null,
"engines_tried": ["rule"],
"engines_failed": []
}
4. ClassificationSource Enum
The ClassificationSource enum (declared in bank_classifier/models.py) documents how the category was determined:
"rule": Determined by a matching rule in the database."ml": Determined by the Logistic Regression classifier."llm": Determined by the Google Gemini API."manual": Assigned via a user manual override in the review dashboard."fallback": Set by system fallback rules (e.g. unclassified OSKO transfers)."unknown": Default state before execution.