Hugging Face's logo

tabularisai
/
ModernFinBERT

Text Classification
Safetensors
English
modernbert
synthetic data
financial-sentiment-analysis
sentiment-analysis
crypto
stocks
finbert
modernfinbert
Synthetic
Model card Files
xet
Community
1
vdmbrsv commited on
Commit
5176999
·
verified ·
1 Parent(s): 4a778fe

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -16
README.md CHANGED
@@ -45,33 +45,35 @@ for i, sentence in enumerate(sentences, 1):
45
  ```
46
 
47
 
48
- ## Benchmark Resuts
49
  | Dataset | Model | Accuracy | F1-Score | Precision | Recall | ROC-AUC |
50
  |---------|-------|----------|----------|-----------|--------|---------|
51
  | FIQA | ModernFinBERT | **0.80** | **0.61** | **0.64** | **0.88** | **0.96** |
52
- | FIQA | distilroberta_financial | _0.54_ | _0.47_ | _0.61_ | _0.71_ | 0.71 |
53
  | FIQA | finbert | 0.48 | 0.43 | 0.59 | 0.66 | 0.76 |
54
- | FIQA | roberta_sentiment | 0.36 | 0.35 | 0.60 | 0.58 | _0.89_ |
55
- | Twitter | ModernFinBERT | 0.71 | _0.70_ | **0.68** | **0.81** | **0.94** |
56
- | Twitter | distilroberta_financial | **0.75** | **0.71** | **0.68** | _0.75_ | _0.87_ |
57
- | Twitter | finbert | _0.73_ | _0.67_ | _0.65_ | _0.70_ | 0.86 |
 
 
58
  | Twitter | roberta_sentiment | 0.70 | 0.61 | 0.63 | 0.60 | 0.82 |
59
  | JeanBaptiste | ModernFinBERT | 0.74 | 0.58 | 0.71 | 0.56 | 0.84 |
60
- | JeanBaptiste | distilroberta_financial | **0.88** | **0.79** | **0.92** | **0.74** | _0.86_ |
61
- | JeanBaptiste | finbert | _0.77_ | _0.68_ | 0.70 | _0.67_ | **0.88** |
62
- | JeanBaptiste | roberta_sentiment | 0.70 | 0.55 | _0.79_ | 0.51 | 0.83 |
 
63
 
64
  ## Model Averages Across All Datasets
65
-
66
  | Model | Accuracy | F1-Score | Precision | Recall | ROC-AUC |
67
  |-------|----------|----------|-----------|--------|---------|
68
- | **ModernFinBERT** | **0.75** | _0.63_ | _0.68_ | **0.75** | **0.91** |
69
- | distilroberta_financial | _0.73_ | **0.66** | **0.73** | _0.73_ | 0.82 |
70
- | finbert | 0.66 | 0.59 | 0.65 | 0.68 | _0.84_ |
71
- | roberta_sentiment | 0.59 | 0.50 | 0.67 | 0.56 | _0.84_ |
 
72
 
73
  ### Legend:
74
-
75
  **Bold** = Best result per metric per dataset
76
 
77
- _Underlined_ = Second best result per metric per dataset
 
45
  ```
46
 
47
 
48
+ ## Benchmark Results
49
  | Dataset | Model | Accuracy | F1-Score | Precision | Recall | ROC-AUC |
50
  |---------|-------|----------|----------|-----------|--------|---------|
51
  | FIQA | ModernFinBERT | **0.80** | **0.61** | **0.64** | **0.88** | **0.96** |
52
+ | FIQA | distilroberta_financial | *0.54* | *0.47* | 0.61 | *0.71* | 0.71 |
53
  | FIQA | finbert | 0.48 | 0.43 | 0.59 | 0.66 | 0.76 |
54
+ | FIQA | finbert-tone | 0.36 | 0.36 | *0.62* | 0.58 | 0.77 |
55
+ | FIQA | roberta_sentiment | 0.36 | 0.35 | 0.60 | 0.58 | *0.89* |
56
+ | Twitter | ModernFinBERT | 0.71 | *0.70* | *0.68* | **0.81** | **0.94** |
57
+ | Twitter | distilroberta_financial | *0.75* | **0.71** | *0.68* | *0.75* | *0.87* |
58
+ | Twitter | finbert-tone | 0.75 | 0.66 | **0.68** | 0.64 | 0.83 |
59
+ | Twitter | finbert | 0.73 | 0.67 | 0.65 | 0.70 | 0.86 |
60
  | Twitter | roberta_sentiment | 0.70 | 0.61 | 0.63 | 0.60 | 0.82 |
61
  | JeanBaptiste | ModernFinBERT | 0.74 | 0.58 | 0.71 | 0.56 | 0.84 |
62
+ | JeanBaptiste | distilroberta_financial | **0.88** | **0.79** | **0.92** | **0.74** | 0.86 |
63
+ | JeanBaptiste | finbert | *0.77* | *0.68* | 0.70 | *0.67* | **0.88** |
64
+ | JeanBaptiste | finbert-tone | 0.74 | 0.60 | 0.72 | 0.56 | *0.86* |
65
+ | JeanBaptiste | roberta_sentiment | 0.70 | 0.55 | *0.79* | 0.51 | 0.83 |
66
 
67
  ## Model Averages Across All Datasets
 
68
  | Model | Accuracy | F1-Score | Precision | Recall | ROC-AUC |
69
  |-------|----------|----------|-----------|--------|---------|
70
+ | **ModernFinBERT** | **0.75** | *0.63* | 0.68 | **0.75** | **0.91** |
71
+ | distilroberta_financial | *0.73* | **0.66** | **0.73** | *0.73* | 0.82 |
72
+ | finbert | 0.66 | 0.59 | 0.65 | 0.68 | *0.84* |
73
+ | finbert-tone | 0.62 | 0.54 | *0.68* | 0.59 | 0.82 |
74
+ | roberta_sentiment | 0.59 | 0.50 | 0.67 | 0.56 | *0.84* |
75
 
76
  ### Legend:
 
77
  **Bold** = Best result per metric per dataset
78
 
79
+ *Italic* = Second best result per metric per dataset