warwickai/fin-perceiver

FINPerceiver

FINPerceiver is a fine-tuned Perceiver IO language model for financial sentiment analysis. More details on the training process of this model are available on the GitHub repository.

Weights & Biases was used to track experiments.

We achieved the following results with 10-fold cross validation.

eval/accuracy  0.8624 (stdev 0.01922)
eval/f1        0.8416 (stdev 0.03738)
eval/loss      0.4314 (stdev 0.05295)
eval/precision 0.8438 (stdev 0.02938)
eval/recall    0.8415 (stdev 0.04458)

The hyperparameters used are as follows.

per_device_train_batch_size  16
per_device_eval_batch_size   16
num_train_epochs             4
learning_rate                2e-5

Datasets

This model was trained on the Financial PhraseBank (>= 50% agreement)