Hugging Face's logo

daytoy-models
/
test

Afar
Venda
Eval Results
Model card Files
xet
Community
daytoy-models commited on
Commit
fc55c6b
·
verified ·
1 Parent(s): 8b9e0b3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +185 -6
README.md CHANGED
@@ -1,4 +1,185 @@
1
- type: nuprl/MultiPL-E
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  name: MultiPL-HumanEval (R)
3
  metrics:
4
  - name: pass@1
@@ -75,11 +256,9 @@
75
  type: pass@1
76
  value: 0.3229
77
  verified: false
78
- extra_gated_prompt: >-
79
- ## Model License Agreement Please read the BigCode [OpenRAIL-M
80
- license](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement)
81
- agreement before accepting it.
82
-
83
  extra_gated_fields:
84
  I accept the above license agreement, and will use the Model complying with the set of use restrictions and sharing requirements: checkbox
85
  language:
 
1
+ ---
2
+ pipeline_tag: text-generation
3
+ inference: true
4
+ widget:
5
+ - text: 'def print_hello_world():'
6
+ example_title: Hello world
7
+ group: Python
8
+ license: bigcode-openrail-m
9
+ datasets:
10
+ - bigcode/the-stack-dedup
11
+ metrics:
12
+ - code_eval
13
+ library_name: transformers
14
+ tags:
15
+ - code
16
+ model-index:
17
+ - name: StarCoder
18
+ results:
19
+ - task:
20
+ type: text-generation
21
+ dataset:
22
+ type: openai_humaneval1
23
+ name: HumanEval1
24
+ metrics:
25
+ - name: pass@1
26
+ type: pass@1
27
+ value: 0.408
28
+ verified: false
29
+ - name: pass@2
30
+ type: pass@2
31
+ value: 0.12345
32
+ verified: false
33
+ - task:
34
+ type: text-generation
35
+ dataset:
36
+ type: openai_humaneval
37
+ name: HumanEval
38
+ metrics:
39
+ - name: pass@1
40
+ type: pass@1
41
+ value:
42
+ dataset:
43
+ type: openai_humaneval
44
+ name: HumanEval
45
+ args: haha
46
+ verified: false
47
+ - name: StarCoder2
48
+ results:
49
+ - task:
50
+ type: text-generation
51
+ dataset:
52
+ type: mbpp
53
+ name: MBPP
54
+ metrics:
55
+ - name: pass@1
56
+ type: pass@1
57
+ value: 0.527
58
+ verified: false
59
+ - task:
60
+ type: text-generation
61
+ dataset:
62
+ type: ds1000
63
+ name: DS-1000 (Overall Completion)
64
+ metrics:
65
+ - name: pass@1
66
+ type: pass@1
67
+ value: 0.26
68
+ verified: false
69
+ - task:
70
+ type: text-generation
71
+ dataset:
72
+ type: nuprl/MultiPL-E
73
+ name: MultiPL-HumanEval (C++)
74
+ metrics:
75
+ - name: pass@1
76
+ type: pass@1
77
+ value: 0.3155
78
+ verified: false
79
+ - task:
80
+ type: text-generation
81
+ dataset:
82
+ type: nuprl/MultiPL-E
83
+ name: MultiPL-HumanEval (C#)
84
+ metrics:
85
+ - name: pass@1
86
+ type: pass@1
87
+ value: 0.2101
88
+ verified: false
89
+ - task:
90
+ type: text-generation
91
+ dataset:
92
+ type: nuprl/MultiPL-E
93
+ name: MultiPL-HumanEval (D)
94
+ metrics:
95
+ - name: pass@1
96
+ type: pass@1
97
+ value: 0.1357
98
+ verified: false
99
+ - task:
100
+ type: text-generation
101
+ dataset:
102
+ type: nuprl/MultiPL-E
103
+ name: MultiPL-HumanEval (Go)
104
+ metrics:
105
+ - name: pass@1
106
+ type: pass@1
107
+ value: 0.1761
108
+ verified: false
109
+ - task:
110
+ type: text-generation
111
+ dataset:
112
+ type: nuprl/MultiPL-E
113
+ name: MultiPL-HumanEval (Java)
114
+ metrics:
115
+ - name: pass@1
116
+ type: pass@1
117
+ value: 0.3022
118
+ verified: false
119
+ - task:
120
+ type: text-generation
121
+ dataset:
122
+ type: nuprl/MultiPL-E
123
+ name: MultiPL-HumanEval (Julia)
124
+ metrics:
125
+ - name: pass@1
126
+ type: pass@1
127
+ value: 0.2302
128
+ verified: false
129
+ - task:
130
+ type: text-generation
131
+ dataset:
132
+ type: nuprl/MultiPL-E
133
+ name: MultiPL-HumanEval (JavaScript)
134
+ metrics:
135
+ - name: pass@1
136
+ type: pass@1
137
+ value: 0.3079
138
+ verified: false
139
+ - task:
140
+ type: text-generation
141
+ dataset:
142
+ type: nuprl/MultiPL-E
143
+ name: MultiPL-HumanEval (Lua)
144
+ metrics:
145
+ - name: pass@1
146
+ type: pass@1
147
+ value: 0.2389
148
+ verified: false
149
+ - task:
150
+ type: text-generation
151
+ dataset:
152
+ type: nuprl/MultiPL-E
153
+ name: MultiPL-HumanEval (PHP)
154
+ metrics:
155
+ - name: pass@1
156
+ type: pass@1
157
+ value: 0.2608
158
+ verified: false
159
+ - task:
160
+ type: text-generation
161
+ dataset:
162
+ type: nuprl/MultiPL-E
163
+ name: MultiPL-HumanEval (Perl)
164
+ metrics:
165
+ - name: pass@1
166
+ type: pass@1
167
+ value: 0.1734
168
+ verified: false
169
+ - task:
170
+ type: text-generation
171
+ dataset:
172
+ type: nuprl/MultiPL-E
173
+ name: MultiPL-HumanEval (Python)
174
+ metrics:
175
+ - name: pass@1
176
+ type: pass@1
177
+ value: 0.3357
178
+ verified: false
179
+ - task:
180
+ type: text-generation
181
+ dataset:
182
+ type: nuprl/MultiPL-E
183
  name: MultiPL-HumanEval (R)
184
  metrics:
185
  - name: pass@1
 
256
  type: pass@1
257
  value: 0.3229
258
  verified: false
259
+ extra_gated_prompt: "## Model License Agreement Please read the BigCode [OpenRAIL-M\
260
+ \ license](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement)\
261
+ \ agreement before accepting it.\n "
 
 
262
  extra_gated_fields:
263
  I accept the above license agreement, and will use the Model complying with the set of use restrictions and sharing requirements: checkbox
264
  language: