Update README.md
Browse files
README.md
CHANGED
|
@@ -17,14 +17,13 @@ Another key feature of our model is that it also avoids lookback bias. In partic
|
|
| 17 |
|
| 18 |
Our model is trained on 1 billion words (1-2 billion tokens) from Parliament Q&As, TV show conversations, music lyrics, patents, FOMC documents, public access books, newspapers, election campaign documents, and research papers. The model is based on the base-size DeBERTa model architecture and a custom ByteLevelBPETokenizer trained using the same training data.
|
| 19 |
|
| 20 |
-
Our model achieves
|
| 21 |
|
| 22 |
| Model | Vocabulary (K) | Backbone #Params (M) | COLA | SST2 | QQP|MNLI|QNLI
|
| 23 |
|------------------------|:--------------:|:--------------------:|:-----------------:|:---------------:|:--------------------:|:-----------------:|:---------------:|
|
| 24 |
-
| ChronoBERT_1999 | 50 | 149 | 0.57|0.92|0.89|0.86|0.91|
|
| 25 |
| FinBERT | 30 | 110 | 0.29|0.89|0.87|0.79|0.86|
|
| 26 |
-
| StoriesLM | 30 | 110 | 0.47
|
| 27 |
-
| NolBERT | 30 | 109 | 0.43
|
| 28 |
|
| 29 |
## Usage Examples
|
| 30 |
|
|
|
|
| 17 |
|
| 18 |
Our model is trained on 1 billion words (1-2 billion tokens) from Parliament Q&As, TV show conversations, music lyrics, patents, FOMC documents, public access books, newspapers, election campaign documents, and research papers. The model is based on the base-size DeBERTa model architecture and a custom ByteLevelBPETokenizer trained using the same training data.
|
| 19 |
|
| 20 |
+
Our model achieves state-of-the-art performance with less than 10% of training data.
|
| 21 |
|
| 22 |
| Model | Vocabulary (K) | Backbone #Params (M) | COLA | SST2 | QQP|MNLI|QNLI
|
| 23 |
|------------------------|:--------------:|:--------------------:|:-----------------:|:---------------:|:--------------------:|:-----------------:|:---------------:|
|
|
|
|
| 24 |
| FinBERT | 30 | 110 | 0.29|0.89|0.87|0.79|0.86|
|
| 25 |
+
| StoriesLM | 30 | 110 | **0.47**|0.90|0.87|0.80|0.87|
|
| 26 |
+
| NolBERT | 30 | 109 | 0.43|**0.91**|**0.91**|**0.82**|**0.89**
|
| 27 |
|
| 28 |
## Usage Examples
|
| 29 |
|