aschaler/bert-bank-transaction-classification

Bank transaction record classification BERT model for domain specific classification application. This FT BERT model specializes the base 'google-bert/bert-base-uncased' on custom synthetic bank transaction data by applying a MLP multi class classification head using Linear Probing then Fine-Tunning (LP-FT).

Method description and dataset

The SFT method unfreezes k final layers of the base model after a warm up step of 3 epochs during FT to improve training and performance of the final classifier. This method minmally extends LP-FT as described in Understanding Linear Probing then Fine-tuning Language Models from NTK Perspective.

Training code loacted at: https://github.com/Andreas3333/model-training/tree/main/bert_for_seq_classification