DoDataThings/distilbert-us-transaction-classifier

Update Notice

A newer version of this model is available: DoDataThings/distilbert-us-transaction-classifier-v2

v2 adds sign-aware classification ([debit]/[credit] prefix), expanded merchant coverage (500+ merchants), PayPal wrapper handling, and a refined 16-category taxonomy. See the v2 model card for details.

This v1 model remains available for backward compatibility. If you don't need sign-aware classification, v1 still works well for basic transaction categorization.

DistilBERT US Bank Transaction Classifier

A fine-tuned DistilBERT model for classifying US bank transaction descriptions into 16 spending categories. Designed as a fallback layer in a multi-tier classification pipeline — not a standalone classifier.

What This Is (and Isn't)

This model was built to handle the long tail of transaction descriptions that merchant rules and bank-provided categories don't cover. It works best as one component in a system:

1. Merchant rules — pattern matching catches known merchants (highest accuracy)
2. Bank-provided categories — map the bank's own classifications (when available)
3. This model — classifies everything else (the fallback)
4. User overrides — manual corrections for edge cases

On its own, this model will make mistakes. Bank transaction descriptions are messy, abbreviated, and sometimes genuinely ambiguous — the same store can be "Groceries" or "Shopping" depending on what was purchased, and no classifier can know that from the description alone.

Training

Fine-tuned on 16,000 synthetic transaction descriptions generated to match real US bank statement formats. The synthetic data covers six format templates (Chase ACH, Apple Card addresses, PayPal prefixes, etc.) with randomized merchants, store numbers, and addresses.

Model:       DistilBERT-base-uncased + LoRA (r=32, alpha=64)
Dataset:     16,000 synthetic samples, 1,000 per category
Trainable:   1.8M / 68.7M parameters (2.6%)
Training:    20 epochs, ~5 minutes on consumer GPU

Loss Curve

Epoch  Train Loss   Val Loss  Train Acc   Val Acc
─────────────────────────────────────────────────
    1      2.670      2.246     17.6%     47.2%
    2      1.685      1.066     60.0%     74.6%
    5      0.355      0.250     90.6%     93.8%
   10      0.086      0.062     97.7%     98.3%
   15      0.036      0.038     99.0%     99.0%
   20      0.028      0.033     99.2%     99.3%

Honest Assessment

The validation accuracy (99.3%) is on synthetic data — the same distribution the model was trained on. Real-world performance is lower because:

• Bank descriptions are noisier than synthetic data. Abbreviations, custom POS prefixes, and institution-specific formats the generator didn't cover.
• Some categories are genuinely ambiguous. WALGREENS is both a pharmacy (Healthcare) and a retail store (Shopping). ABC Stores in Hawaii sells groceries and souvenirs. The "correct" answer depends on what was purchased.
• The model doesn't know every merchant. It learned patterns and common names, but niche local businesses or new companies won't match training data.
• ACH-format transactions are hard. INSTITUTION PURPOSE PPD ID: looks the same whether it's insurance, utilities, or a subscription. The institution name is the only signal, and the model's vocabulary here is limited.

In our testing against ~1,800 real transactions with bank/rule-derived labels, the model's raw accuracy was ~85%. However, about 6% of the "ground truth" labels were actually wrong (bank miscategorizations), and the model correctly overrode them — adjusted accuracy is ~92%.

Categories (16)

Category	What it covers
Restaurants	Fast food, sit-down, coffee shops, food delivery
Groceries	Supermarkets, warehouse clubs, farmers markets
Shopping	Retail, online purchases, department stores
Transportation	Gas, rideshare, auto maintenance, parking
Entertainment	Movies, events, gaming
Utilities	Electric, internet, phone, water
Subscription	Streaming, SaaS, news, software
Healthcare	Pharmacy, doctor, dentist, therapy
Insurance	Auto, home, health, life insurance
Housing	Rent, mortgage, home maintenance
Travel	Hotels, airlines, car rental
Education	Online courses, books, tuition
Personal Care	Salon, gym, beauty, spa
Transfer	CC autopay, Zelle/Venmo sends, bank transfers
Income	Payroll, direct deposit, interest, refunds
Fees	Bank fees, late fees, service charges

"Business" is intentionally not a category. Whether a transaction is a business expense depends on which account it's charged to, not the description. That's an account-level annotation, not a classification task.

Usage

Python

from transformers import pipeline

classifier = pipeline("text-classification", model="DoDataThings/distilbert-us-transaction-classifier")

result = classifier("STARBUCKS #1234")
print(result)  # [{'label': 'Restaurants', 'score': 0.93}]

JavaScript (Transformers.js)

const { pipeline } = require('@xenova/transformers');

const classifier = await pipeline(
  'text-classification',
  'DoDataThings/distilbert-us-transaction-classifier'
);

const result = await classifier('STARBUCKS #1234');
// [{ label: 'Restaurants', score: 0.93 }]

An ONNX export is included in the onnx/ subdirectory.

Training Data

The synthetic dataset is published at DoDataThings/us-bank-transaction-categories. The generator script is open source — you can extend the merchant pools, add format templates, or increase sample counts to improve accuracy for your use case.

Limitations

• US bank formats only. Trained on Chase, Apple Card, PayPal, and Capital One statement patterns. International bank descriptions will not classify well.
• Synthetic training data. The model learned from generated descriptions, not real transaction data. It may miss patterns that only appear in real bank exports.
• Shopping is the weakest category (~92% on synthetic) due to overlap with Subscription (Amazon sells both products and subscriptions) and Groceries (warehouse clubs).
• Not a standalone solution. This model is a fallback layer. For production use, pair it with merchant rules and bank category mappings for best results.

License

Apache 2.0