Hugging Face's logo

theprint
/
Code-Llama-Bagel-8B

Transformers
Safetensors
GGUF
llama
Merge
mergekit
lazymergekit
theprint/Code-Llama-Bagel-8B
ajibawa-2023/Code-Llama-3-8B
jondurbin/bagel-8b-v1.0
Eval Results (legacy)
text-generation-inference
conversational
Model card Files
xet
 Community
2

Adding Evaluation Results

#1
by leaderboard-pr-bot - opened
base: refs/heads/main
←
from: refs/pr/1
Discussion Files changed
+113
-5
Files changed (1) hide show
  1. README.md +113 -5
README.md CHANGED
@@ -1,7 +1,5 @@
1
  ---
2
- base_model:
3
- - ajibawa-2023/Code-Llama-3-8B
4
- - jondurbin/bagel-8b-v1.0
5
  tags:
6
  - merge
7
  - mergekit
@@ -9,7 +7,104 @@ tags:
9
  - theprint/Code-Llama-Bagel-8B
10
  - ajibawa-2023/Code-Llama-3-8B
11
  - jondurbin/bagel-8b-v1.0
12
- license: llama3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
  ---
14
 
15
  # Code-Llama-Bagel-8B
@@ -62,4 +157,17 @@ pipeline = transformers.pipeline(
62
 
63
  outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
64
  print(outputs[0]["generated_text"])
65
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: llama3
 
 
3
  tags:
4
  - merge
5
  - mergekit
 
7
  - theprint/Code-Llama-Bagel-8B
8
  - ajibawa-2023/Code-Llama-3-8B
9
  - jondurbin/bagel-8b-v1.0
10
+ base_model:
11
+ - ajibawa-2023/Code-Llama-3-8B
12
+ - jondurbin/bagel-8b-v1.0
13
+ model-index:
14
+ - name: Code-Llama-Bagel-8B
15
+ results:
16
+ - task:
17
+ type: text-generation
18
+ name: Text Generation
19
+ dataset:
20
+ name: IFEval (0-Shot)
21
+ type: HuggingFaceH4/ifeval
22
+ args:
23
+ num_few_shot: 0
24
+ metrics:
25
+ - type: inst_level_strict_acc and prompt_level_strict_acc
26
+ value: 25.3
27
+ name: strict accuracy
28
+ source:
29
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=theprint/Code-Llama-Bagel-8B
30
+ name: Open LLM Leaderboard
31
+ - task:
32
+ type: text-generation
33
+ name: Text Generation
34
+ dataset:
35
+ name: BBH (3-Shot)
36
+ type: BBH
37
+ args:
38
+ num_few_shot: 3
39
+ metrics:
40
+ - type: acc_norm
41
+ value: 25.34
42
+ name: normalized accuracy
43
+ source:
44
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=theprint/Code-Llama-Bagel-8B
45
+ name: Open LLM Leaderboard
46
+ - task:
47
+ type: text-generation
48
+ name: Text Generation
49
+ dataset:
50
+ name: MATH Lvl 5 (4-Shot)
51
+ type: hendrycks/competition_math
52
+ args:
53
+ num_few_shot: 4
54
+ metrics:
55
+ - type: exact_match
56
+ value: 4.98
57
+ name: exact match
58
+ source:
59
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=theprint/Code-Llama-Bagel-8B
60
+ name: Open LLM Leaderboard
61
+ - task:
62
+ type: text-generation
63
+ name: Text Generation
64
+ dataset:
65
+ name: GPQA (0-shot)
66
+ type: Idavidrein/gpqa
67
+ args:
68
+ num_few_shot: 0
69
+ metrics:
70
+ - type: acc_norm
71
+ value: 3.47
72
+ name: acc_norm
73
+ source:
74
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=theprint/Code-Llama-Bagel-8B
75
+ name: Open LLM Leaderboard
76
+ - task:
77
+ type: text-generation
78
+ name: Text Generation
79
+ dataset:
80
+ name: MuSR (0-shot)
81
+ type: TAUR-Lab/MuSR
82
+ args:
83
+ num_few_shot: 0
84
+ metrics:
85
+ - type: acc_norm
86
+ value: 7.53
87
+ name: acc_norm
88
+ source:
89
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=theprint/Code-Llama-Bagel-8B
90
+ name: Open LLM Leaderboard
91
+ - task:
92
+ type: text-generation
93
+ name: Text Generation
94
+ dataset:
95
+ name: MMLU-PRO (5-shot)
96
+ type: TIGER-Lab/MMLU-Pro
97
+ config: main
98
+ split: test
99
+ args:
100
+ num_few_shot: 5
101
+ metrics:
102
+ - type: acc
103
+ value: 20.24
104
+ name: accuracy
105
+ source:
106
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=theprint/Code-Llama-Bagel-8B
107
+ name: Open LLM Leaderboard
108
  ---
109
 
110
  # Code-Llama-Bagel-8B
 
157
 
158
  outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
159
  print(outputs[0]["generated_text"])
160
+ ```
161
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
162
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_theprint__Code-Llama-Bagel-8B)
163
+
164
+ | Metric |Value|
165
+ |-------------------|----:|
166
+ |Avg. |14.48|
167
+ |IFEval (0-Shot) |25.30|
168
+ |BBH (3-Shot) |25.34|
169
+ |MATH Lvl 5 (4-Shot)| 4.98|
170
+ |GPQA (0-shot) | 3.47|
171
+ |MuSR (0-shot) | 7.53|
172
+ |MMLU-PRO (5-shot) |20.24|
173
+