Evaluation results for adit94/nlpcharade model as a base model for other tasks
#1
by
eladven
- opened
README.md
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# adit94/nlpcharade model
|
| 2 |
+
This model is based on t5-base pretrained model.
|
| 3 |
+
|
| 4 |
+
|
| 5 |
+
## Model Recycling
|
| 6 |
+
|
| 7 |
+
[Evaluation on 36 datasets](https://ibm.github.io/model-recycling/model_gain_chart?avg=2.78&mnli_lp=nan&20_newsgroup=-29.01&ag_news=2.38&amazon_reviews_multi=4.40&anli=1.58&boolq=10.84&cb=-8.92&cola=-2.62&copa=39.82&dbpedia=12.81&esnli=0.60&financial_phrasebank=1.31&imdb=-10.84&isear=26.32&mnli=8.64&mrpc=3.06&multirc=12.08&poem_sentiment=-29.04&qnli=-34.05&qqp=1.74&rotten_tomatoes=-36.72&rte=16.64&sst2=-9.88&sst_5bins=18.68&stsb=-5.99&trec_coarse=-30.77&trec_fine=-0.01&tweet_ev_emoji=47.56&tweet_ev_emotion=10.81&tweet_ev_hate=21.50&tweet_ev_irony=10.21&tweet_ev_offensive=-13.09&tweet_ev_sentiment=16.40&wic=4.61&wnli=0.99&wsc=17.17&yahoo_answers=21.01&model_name=adit94%2Fnlpcharade&base_name=t5-base) using adit94/nlpcharade as a base model yields average score of 78.23 in comparison to 75.45 by t5-base.
|
| 8 |
+
|
| 9 |
+
The model is ranked 1st among all tested models for the t5-base architecture as of 21/12/2022
|
| 10 |
+
Results:
|
| 11 |
+
|
| 12 |
+
| 20_newsgroup | ag_news | amazon_reviews_multi | anli | boolq | cb | cola | copa | dbpedia | esnli | financial_phrasebank | imdb | isear | mnli | mrpc | multirc | poem_sentiment | qnli | qqp | rotten_tomatoes | rte | sst2 | sst_5bins | stsb | trec_coarse | trec_fine | tweet_ev_emoji | tweet_ev_emotion | tweet_ev_hate | tweet_ev_irony | tweet_ev_offensive | tweet_ev_sentiment | wic | wnli | wsc | yahoo_answers |
|
| 13 |
+
|---------------:|----------:|-----------------------:|-------:|--------:|--------:|--------:|--------:|----------:|--------:|-----------------------:|--------:|--------:|--------:|--------:|----------:|-----------------:|-------:|--------:|------------------:|--------:|--------:|------------:|--------:|--------------:|------------:|-----------------:|-------------------:|----------------:|-----------------:|---------------------:|---------------------:|------:|-------:|--------:|----------------:|
|
| 14 |
+
| 56.1086 | 91.8 | 70.9459 | 48.625 | 87.5 | 66.6144 | 79.2905 | 89.4667 | 89.212 | 90.3196 | 86.6151 | 81.4919 | 97.6 | 92.4401 | 88.7255 | 72.3598 | 45.38 | 56.338 | 90.6752 | 51.8855 | 90.3196 | 83.9535 | 74.2347 | 79.3272 | 66.44 | 92.3165 | 92.4401 | 90.3196 | 74.2347 | 83.9535 | 70.9459 | 86.6151 | 71.8 | 56.338 | 77.1667 | 92.596 |
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
For more information, see: [Model Recycling](https://ibm.github.io/model-recycling/)
|