Update README.md
Browse files
README.md
CHANGED
|
@@ -24,13 +24,17 @@ The model is fine-tuned and tested on the natural language inference (NLI) datas
|
|
| 24 |
|
| 25 |
Below is a confusion matrix calculated on zero-shot inferences for the 10 most popular categories in the Test split of [reddgr/nli-chatbot-prompt-categorization](https://huggingface.co/datasets/reddgr/nli-chatbot-prompt-categorization) at the time of the first model upload. The classification with the base model on the same small test dataset is shown for comparison:
|
| 26 |
|
| 27 |
-
 by
|
| 30 |
|
| 31 |
-
The chart below compares the results for the 12 most popular candidate classes in the Test split, where the base model's zero-shot accuracy is outperformed by
|
| 32 |
|
| 33 |
-

|
| 36 |
|
|
|
|
| 24 |
|
| 25 |
Below is a confusion matrix calculated on zero-shot inferences for the 10 most popular categories in the Test split of [reddgr/nli-chatbot-prompt-categorization](https://huggingface.co/datasets/reddgr/nli-chatbot-prompt-categorization) at the time of the first model upload. The classification with the base model on the same small test dataset is shown for comparison:
|
| 26 |
|
| 27 |
+

|
| 28 |
|
| 29 |
+
The current version of the fine-tuned model outperforms the base model [facebook/bart-large-mnli](https://huggingface.co/facebook/bart-large-mnli) by 34 percentage points (76% accuracy vs 42% accuracy) in a test set with 10 candidate zero-shot classes (the most frequent categories in the test split of [reddgr/nli-chatbot-prompt-categorization](https://huggingface.co/datasets/reddgr/nli-chatbot-prompt-categorization)).
|
| 30 |
|
| 31 |
+
The chart below compares the results for the 12 most popular candidate classes in the Test split, where the base model's zero-shot accuracy is outperformed by 32 percentage points:
|
| 32 |
|
| 33 |
+

|
| 34 |
+
|
| 35 |
+
We can also use the model to perform zero-shot inferences on combinations of categories formulated in natural language. The chart below compares the results for the 6 main category groups that classify conversations in [Talking to Chatbots](https://talkingtochatbots.com)
|
| 36 |
+
|
| 37 |
+

|
| 38 |
|
| 39 |
The dataset and the model are continously updated as they assist with content publishing on my website [Talking to Chatbots](https://talkingtochatbots)
|
| 40 |
|