reddgr commited on
Commit
4fcb467
·
verified ·
1 Parent(s): 4238b9d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -4
README.md CHANGED
@@ -24,13 +24,17 @@ The model is fine-tuned and tested on the natural language inference (NLI) datas
24
 
25
  Below is a confusion matrix calculated on zero-shot inferences for the 10 most popular categories in the Test split of [reddgr/nli-chatbot-prompt-categorization](https://huggingface.co/datasets/reddgr/nli-chatbot-prompt-categorization) at the time of the first model upload. The classification with the base model on the same small test dataset is shown for comparison:
26
 
27
- ![Zero-shot prompt classification confusion matrix for reddgr/zero-shot-prompt-classifier-bart-ft](https://talkingtochatbots.com/wp-content/uploads/2025/01/zero-shot-prompt-classification-comparison-10-classes-60-accuracy.png)
28
 
29
- The current version of the fine-tuned model outperforms the base model [facebook/bart-large-mnli](https://huggingface.co/facebook/bart-large-mnli) by 24 percentage points (60% accuracy vs 36% accuracy) in a test set with 10 candidate zero-shot classes (the most frequent categories in the test split of [reddgr/nli-chatbot-prompt-categorization](https://huggingface.co/datasets/reddgr/nli-chatbot-prompt-categorization)).
30
 
31
- The chart below compares the results for the 12 most popular candidate classes in the Test split, where the base model's zero-shot accuracy is outperformed by 25 percentage points:
32
 
33
- ![Zero-shot prompt classification confusion matrix for reddgr/zero-shot-prompt-classifier-bart-ft](https://talkingtochatbots.com/wp-content/uploads/2025/01/zero-shot-prompt-classification-comparison-12-classes-57-accuracy.png)
 
 
 
 
34
 
35
  The dataset and the model are continously updated as they assist with content publishing on my website [Talking to Chatbots](https://talkingtochatbots)
36
 
 
24
 
25
  Below is a confusion matrix calculated on zero-shot inferences for the 10 most popular categories in the Test split of [reddgr/nli-chatbot-prompt-categorization](https://huggingface.co/datasets/reddgr/nli-chatbot-prompt-categorization) at the time of the first model upload. The classification with the base model on the same small test dataset is shown for comparison:
26
 
27
+ ![Zero-shot prompt classification confusion matrix for reddgr/zero-shot-prompt-classifier-bart-ft](https://talkingtochatbots.com/wp-content/uploads/zero-shot-prompt-classifier/10-category-matrix.png)
28
 
29
+ The current version of the fine-tuned model outperforms the base model [facebook/bart-large-mnli](https://huggingface.co/facebook/bart-large-mnli) by 34 percentage points (76% accuracy vs 42% accuracy) in a test set with 10 candidate zero-shot classes (the most frequent categories in the test split of [reddgr/nli-chatbot-prompt-categorization](https://huggingface.co/datasets/reddgr/nli-chatbot-prompt-categorization)).
30
 
31
+ The chart below compares the results for the 12 most popular candidate classes in the Test split, where the base model's zero-shot accuracy is outperformed by 32 percentage points:
32
 
33
+ ![Zero-shot prompt classification confusion matrix for reddgr/zero-shot-prompt-classifier-bart-ft](https://talkingtochatbots.com/wp-content/uploads/zero-shot-prompt-classifier/12-category-matrix.png)
34
+
35
+ We can also use the model to perform zero-shot inferences on combinations of categories formulated in natural language. The chart below compares the results for the 6 main category groups that classify conversations in [Talking to Chatbots](https://talkingtochatbots.com)
36
+
37
+ ![Zero-shot prompt classification confusion matrix for reddgr/zero-shot-prompt-classifier-bart-ft](https://talkingtochatbots.com/wp-content/uploads/zero-shot-prompt-classifier/6-category-matrix.png)
38
 
39
  The dataset and the model are continously updated as they assist with content publishing on my website [Talking to Chatbots](https://talkingtochatbots)
40