Update README.md
Browse files
README.md
CHANGED
|
@@ -92,6 +92,19 @@ Despite its extremely constrained compute budget (a fraction of the resources us
|
|
| 92 |
|
| 93 |
While the benchmarks in the previous section utilized Linear Probing to test frozen embeddings, the [TabiBench](https://github.com/boun-tabi-LMG/Tabibert) suite evaluates the model's performance under Full Fine-Tuning conditions. This provides a clear picture of tifilBERT's adaptability as a pre-trained backbone when its entire weight architecture is optimized for a specific Turkish NLP challenge.
|
| 94 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 95 |
### Set 1 (Academic / Medical Text Classification) - Metric: Macro F1
|
| 96 |
|
| 97 |
| Dataset | Number of Samples | tifilBERT-Base | BERTurk | TabiBERT |
|
|
@@ -144,19 +157,6 @@ While the benchmarks in the previous section utilized Linear Probing to test fro
|
|
| 144 |
| ProductReviews | 35,275 | 0.8515 | 0.8430 | 0.8432 |
|
| 145 |
| **Weighted Average** | **37,675** | **0.8417** | **0.8342** | **0.8344** |
|
| 146 |
|
| 147 |
-
<br>
|
| 148 |
-
|
| 149 |
-
### 🎯 General Summary Table (Comparison of All Task Groups)
|
| 150 |
-
*A collective view of the weighted averages of all models across the 5 main task groups:*
|
| 151 |
-
|
| 152 |
-
| Task Group | Metric | tifilBERT-Base | BERTurk | TabiBERT |
|
| 153 |
-
| :--- | :--- | :--- | :--- | :--- |
|
| 154 |
-
| Set 1 (Medical/Academic) | Macro F1 | 69.70% | 70.90% | **71.91%** |
|
| 155 |
-
| Set 2 (NLI) | Macro F1 | 83.87% | 84.33% | **84.51%** |
|
| 156 |
-
| Set 3 (QA) | F1 Score | **77.14%** | 60.16% | 69.71% |
|
| 157 |
-
| Set 4 (STS) | Pearson | **86.00%** | 85.33% | 84.75% |
|
| 158 |
-
| Set 5 (General Classification) | Macro F1 | **84.17%** | 83.42% | 83.44% |
|
| 159 |
-
|
| 160 |
### 💡 Key Insights & Takeaways from TabiBench
|
| 161 |
|
| 162 |
The Full Fine-Tuning results on TabiBench perfectly illustrate the success of tifilBERT's non-traditional training pipeline:
|
|
|
|
| 92 |
|
| 93 |
While the benchmarks in the previous section utilized Linear Probing to test frozen embeddings, the [TabiBench](https://github.com/boun-tabi-LMG/Tabibert) suite evaluates the model's performance under Full Fine-Tuning conditions. This provides a clear picture of tifilBERT's adaptability as a pre-trained backbone when its entire weight architecture is optimized for a specific Turkish NLP challenge.
|
| 94 |
|
| 95 |
+
### 🎯 General Summary Table (Comparison of All Task Groups)
|
| 96 |
+
*A collective view of the weighted averages of all models across the 5 main task groups:*
|
| 97 |
+
|
| 98 |
+
| Task Group | Metric | tifilBERT-Base | BERTurk | TabiBERT |
|
| 99 |
+
| :--- | :--- | :--- | :--- | :--- |
|
| 100 |
+
| Set 1 (Medical/Academic) | Macro F1 | 69.70% | 70.90% | **71.91%** |
|
| 101 |
+
| Set 2 (NLI) | Macro F1 | 83.87% | 84.33% | **84.51%** |
|
| 102 |
+
| Set 3 (QA) | F1 Score | **77.14%** | 60.16% | 69.71% |
|
| 103 |
+
| Set 4 (STS) | Pearson | **86.00%** | 85.33% | 84.75% |
|
| 104 |
+
| Set 5 (General Classification) | Macro F1 | **84.17%** | 83.42% | 83.44% |
|
| 105 |
+
|
| 106 |
+
<br>
|
| 107 |
+
|
| 108 |
### Set 1 (Academic / Medical Text Classification) - Metric: Macro F1
|
| 109 |
|
| 110 |
| Dataset | Number of Samples | tifilBERT-Base | BERTurk | TabiBERT |
|
|
|
|
| 157 |
| ProductReviews | 35,275 | 0.8515 | 0.8430 | 0.8432 |
|
| 158 |
| **Weighted Average** | **37,675** | **0.8417** | **0.8342** | **0.8344** |
|
| 159 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 160 |
### 💡 Key Insights & Takeaways from TabiBench
|
| 161 |
|
| 162 |
The Full Fine-Tuning results on TabiBench perfectly illustrate the success of tifilBERT's non-traditional training pipeline:
|