Hugging Face's logo

Deathsquad10
/
TinyLlama-repeat

Text Generation
Transformers
Safetensors
English
llama
conversational
text-generation-inference
Model card Files
xet
 Community
1

Adding Evaluation Results

#1
by leaderboard-pr-bot - opened
base: refs/heads/main
←
from: refs/pr/1
Discussion Files changed
+131
-4
Files changed (1) hide show
  1. README.md +131 -4
README.md CHANGED
@@ -1,14 +1,128 @@
1
  ---
 
 
2
  license: apache-2.0
3
  datasets:
4
  - cerebras/SlimPajama-627B
5
  - bigcode/starcoderdata
6
  - HuggingFaceH4/ultrachat_200k
7
  - HuggingFaceH4/ultrafeedback_binarized
8
- language:
9
- - en
10
  widget:
11
- - text: "<|system|>\nYou are a chatbot who can help code!</s>\n<|user|>\nWrite out the first 10 digits of the fibonacci sequence in Python and print it out to the CLI.</s>\n<|assistant|>\n"
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  ---
13
  <div align="center">
14
 
@@ -71,4 +185,17 @@ print(outputs[0]["generated_text"])
71
  # How many helicopters can a human eat in one sitting?</s>
72
  # <|assistant|>
73
  # ...
74
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
  license: apache-2.0
5
  datasets:
6
  - cerebras/SlimPajama-627B
7
  - bigcode/starcoderdata
8
  - HuggingFaceH4/ultrachat_200k
9
  - HuggingFaceH4/ultrafeedback_binarized
 
 
10
  widget:
11
+ - text: '<|system|>
12
+
13
+ You are a chatbot who can help code!</s>
14
+
15
+ <|user|>
16
+
17
+ Write out the first 10 digits of the fibonacci sequence in Python and print it
18
+ out to the CLI.</s>
19
+
20
+ <|assistant|>
21
+
22
+ '
23
+ model-index:
24
+ - name: TinyLlama-repeat
25
+ results:
26
+ - task:
27
+ type: text-generation
28
+ name: Text Generation
29
+ dataset:
30
+ name: AI2 Reasoning Challenge (25-Shot)
31
+ type: ai2_arc
32
+ config: ARC-Challenge
33
+ split: test
34
+ args:
35
+ num_few_shot: 25
36
+ metrics:
37
+ - type: acc_norm
38
+ value: 35.24
39
+ name: normalized accuracy
40
+ source:
41
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Deathsquad10/TinyLlama-repeat
42
+ name: Open LLM Leaderboard
43
+ - task:
44
+ type: text-generation
45
+ name: Text Generation
46
+ dataset:
47
+ name: HellaSwag (10-Shot)
48
+ type: hellaswag
49
+ split: validation
50
+ args:
51
+ num_few_shot: 10
52
+ metrics:
53
+ - type: acc_norm
54
+ value: 60.25
55
+ name: normalized accuracy
56
+ source:
57
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Deathsquad10/TinyLlama-repeat
58
+ name: Open LLM Leaderboard
59
+ - task:
60
+ type: text-generation
61
+ name: Text Generation
62
+ dataset:
63
+ name: MMLU (5-Shot)
64
+ type: cais/mmlu
65
+ config: all
66
+ split: test
67
+ args:
68
+ num_few_shot: 5
69
+ metrics:
70
+ - type: acc
71
+ value: 26.07
72
+ name: accuracy
73
+ source:
74
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Deathsquad10/TinyLlama-repeat
75
+ name: Open LLM Leaderboard
76
+ - task:
77
+ type: text-generation
78
+ name: Text Generation
79
+ dataset:
80
+ name: TruthfulQA (0-shot)
81
+ type: truthful_qa
82
+ config: multiple_choice
83
+ split: validation
84
+ args:
85
+ num_few_shot: 0
86
+ metrics:
87
+ - type: mc2
88
+ value: 38.78
89
+ source:
90
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Deathsquad10/TinyLlama-repeat
91
+ name: Open LLM Leaderboard
92
+ - task:
93
+ type: text-generation
94
+ name: Text Generation
95
+ dataset:
96
+ name: Winogrande (5-shot)
97
+ type: winogrande
98
+ config: winogrande_xl
99
+ split: validation
100
+ args:
101
+ num_few_shot: 5
102
+ metrics:
103
+ - type: acc
104
+ value: 60.46
105
+ name: accuracy
106
+ source:
107
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Deathsquad10/TinyLlama-repeat
108
+ name: Open LLM Leaderboard
109
+ - task:
110
+ type: text-generation
111
+ name: Text Generation
112
+ dataset:
113
+ name: GSM8k (5-shot)
114
+ type: gsm8k
115
+ config: main
116
+ split: test
117
+ args:
118
+ num_few_shot: 5
119
+ metrics:
120
+ - type: acc
121
+ value: 1.74
122
+ name: accuracy
123
+ source:
124
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Deathsquad10/TinyLlama-repeat
125
+ name: Open LLM Leaderboard
126
  ---
127
  <div align="center">
128
 
 
185
  # How many helicopters can a human eat in one sitting?</s>
186
  # <|assistant|>
187
  # ...
188
+ ```
189
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
190
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Deathsquad10__TinyLlama-repeat)
191
+
192
+ | Metric |Value|
193
+ |---------------------------------|----:|
194
+ |Avg. |37.09|
195
+ |AI2 Reasoning Challenge (25-Shot)|35.24|
196
+ |HellaSwag (10-Shot) |60.25|
197
+ |MMLU (5-Shot) |26.07|
198
+ |TruthfulQA (0-shot) |38.78|
199
+ |Winogrande (5-shot) |60.46|
200
+ |GSM8k (5-shot) | 1.74|
201
+