Hugging Face's logo

Mohammed-Altaf
/
Medical-ChatBot

Text Generation
Transformers
PyTorch
English
gpt2
medical
text-generation-inference
Model card Files
xet
 Community
6
leaderboard-pr-bot commited on
Commit
d442940
·
verified ·
1 Parent(s): 9e2d5d7

Adding Evaluation Results

Browse files

This is an automated PR created with https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr

The purpose of this PR is to add evaluation results from the Open LLM Leaderboard to your model card.

If you encounter any issues, please report them to https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr/discussions

Files changed (1) hide show
  1. README.md +121 -5
README.md CHANGED
@@ -1,12 +1,115 @@
1
  ---
 
 
 
 
2
  tags:
3
  - medical
4
- license: mit
5
  datasets:
6
  - Mohammed-Altaf/medical-instruction-120k
7
- language:
8
- - en
9
- library_name: transformers
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
  ---
11
  Please note that the chatbot is designed for research purposes only and is not intended for use in real medical settings. While the chatbot has been trained to provide accurate and helpful responses, it is not a substitute for professional medical advice, diagnosis, or treatment. The information provided by the chatbot should not be used to make medical decisions, and any health concerns should be addressed by a licensed healthcare provider.
12
 
@@ -53,4 +156,17 @@ The conversation between human and AI assistant.
53
  The conversation between human and AI assistant.
54
  [|Human|] what type of honey is best for a bad covid cough?
55
  [|AI|] Manuka honey has been shown to have anti-inflammatory and antibacterial properties that can help alleviate symptoms of a bad covid cough.
56
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
+ license: mit
5
+ library_name: transformers
6
  tags:
7
  - medical
 
8
  datasets:
9
  - Mohammed-Altaf/medical-instruction-120k
10
+ model-index:
11
+ - name: Medical-ChatBot
12
+ results:
13
+ - task:
14
+ type: text-generation
15
+ name: Text Generation
16
+ dataset:
17
+ name: AI2 Reasoning Challenge (25-Shot)
18
+ type: ai2_arc
19
+ config: ARC-Challenge
20
+ split: test
21
+ args:
22
+ num_few_shot: 25
23
+ metrics:
24
+ - type: acc_norm
25
+ value: 30.55
26
+ name: normalized accuracy
27
+ source:
28
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Mohammed-Altaf/Medical-ChatBot
29
+ name: Open LLM Leaderboard
30
+ - task:
31
+ type: text-generation
32
+ name: Text Generation
33
+ dataset:
34
+ name: HellaSwag (10-Shot)
35
+ type: hellaswag
36
+ split: validation
37
+ args:
38
+ num_few_shot: 10
39
+ metrics:
40
+ - type: acc_norm
41
+ value: 38.63
42
+ name: normalized accuracy
43
+ source:
44
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Mohammed-Altaf/Medical-ChatBot
45
+ name: Open LLM Leaderboard
46
+ - task:
47
+ type: text-generation
48
+ name: Text Generation
49
+ dataset:
50
+ name: MMLU (5-Shot)
51
+ type: cais/mmlu
52
+ config: all
53
+ split: test
54
+ args:
55
+ num_few_shot: 5
56
+ metrics:
57
+ - type: acc
58
+ value: 25.98
59
+ name: accuracy
60
+ source:
61
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Mohammed-Altaf/Medical-ChatBot
62
+ name: Open LLM Leaderboard
63
+ - task:
64
+ type: text-generation
65
+ name: Text Generation
66
+ dataset:
67
+ name: TruthfulQA (0-shot)
68
+ type: truthful_qa
69
+ config: multiple_choice
70
+ split: validation
71
+ args:
72
+ num_few_shot: 0
73
+ metrics:
74
+ - type: mc2
75
+ value: 41.25
76
+ source:
77
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Mohammed-Altaf/Medical-ChatBot
78
+ name: Open LLM Leaderboard
79
+ - task:
80
+ type: text-generation
81
+ name: Text Generation
82
+ dataset:
83
+ name: Winogrande (5-shot)
84
+ type: winogrande
85
+ config: winogrande_xl
86
+ split: validation
87
+ args:
88
+ num_few_shot: 5
89
+ metrics:
90
+ - type: acc
91
+ value: 55.41
92
+ name: accuracy
93
+ source:
94
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Mohammed-Altaf/Medical-ChatBot
95
+ name: Open LLM Leaderboard
96
+ - task:
97
+ type: text-generation
98
+ name: Text Generation
99
+ dataset:
100
+ name: GSM8k (5-shot)
101
+ type: gsm8k
102
+ config: main
103
+ split: test
104
+ args:
105
+ num_few_shot: 5
106
+ metrics:
107
+ - type: acc
108
+ value: 0.99
109
+ name: accuracy
110
+ source:
111
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Mohammed-Altaf/Medical-ChatBot
112
+ name: Open LLM Leaderboard
113
  ---
114
  Please note that the chatbot is designed for research purposes only and is not intended for use in real medical settings. While the chatbot has been trained to provide accurate and helpful responses, it is not a substitute for professional medical advice, diagnosis, or treatment. The information provided by the chatbot should not be used to make medical decisions, and any health concerns should be addressed by a licensed healthcare provider.
115
 
 
156
  The conversation between human and AI assistant.
157
  [|Human|] what type of honey is best for a bad covid cough?
158
  [|AI|] Manuka honey has been shown to have anti-inflammatory and antibacterial properties that can help alleviate symptoms of a bad covid cough.
159
+ ```
160
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
161
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Mohammed-Altaf__Medical-ChatBot)
162
+
163
+ | Metric |Value|
164
+ |---------------------------------|----:|
165
+ |Avg. |32.13|
166
+ |AI2 Reasoning Challenge (25-Shot)|30.55|
167
+ |HellaSwag (10-Shot) |38.63|
168
+ |MMLU (5-Shot) |25.98|
169
+ |TruthfulQA (0-shot) |41.25|
170
+ |Winogrande (5-shot) |55.41|
171
+ |GSM8k (5-shot) | 0.99|
172
+