Dzluck commited on
Commit
bb93c16
·
verified ·
1 Parent(s): 1b87cd5

Add files using upload-large-folder tool

Browse files
.gitattributes CHANGED
@@ -33,3 +33,17 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ stable-code-instruct-3b.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ stable-code-instruct-3b.Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
38
+ stable-code-instruct-3b.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
39
+ stable-code-instruct-3b.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
40
+ stable-code-instruct-3b.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
41
+ stable-code-instruct-3b.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ stable-code-instruct-3b.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
43
+ stable-code-instruct-3b.Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ stable-code-instruct-3b.Q5_1.gguf filter=lfs diff=lfs merge=lfs -text
45
+ stable-code-instruct-3b.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
46
+ stable-code-instruct-3b.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
47
+ stable-code-instruct-3b.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
48
+ stable-code-instruct-3b.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
49
+ stable-code-instruct-3b.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,188 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ language:
4
+ - en
5
+ tags:
6
+ - causal-lm
7
+ - code
8
+ base_model: stabilityai/stable-code-instruct-3b
9
+ metrics:
10
+ - code_eval
11
+ library_name: transformers
12
+ model-index:
13
+ - name: stabilityai/stable-code-instruct-3b
14
+ results:
15
+ - task:
16
+ type: text-generation
17
+ dataset:
18
+ type: nuprl/MultiPL-E
19
+ name: MultiPL-HumanEval (Python)
20
+ metrics:
21
+ - name: pass@1
22
+ type: pass@1
23
+ value: 32.4
24
+ verified: false
25
+ - task:
26
+ type: text-generation
27
+ dataset:
28
+ type: nuprl/MultiPL-E
29
+ name: MultiPL-HumanEval (C++)
30
+ metrics:
31
+ - name: pass@1
32
+ type: pass@1
33
+ value: 30.9
34
+ verified: false
35
+ - task:
36
+ type: text-generation
37
+ dataset:
38
+ type: nuprl/MultiPL-E
39
+ name: MultiPL-HumanEval (Java)
40
+ metrics:
41
+ - name: pass@1
42
+ type: pass@1
43
+ value: 32.1
44
+ verified: false
45
+ - task:
46
+ type: text-generation
47
+ dataset:
48
+ type: nuprl/MultiPL-E
49
+ name: MultiPL-HumanEval (JavaScript)
50
+ metrics:
51
+ - name: pass@1
52
+ type: pass@1
53
+ value: 32.1
54
+ verified: false
55
+ - task:
56
+ type: text-generation
57
+ dataset:
58
+ type: nuprl/MultiPL-E
59
+ name: MultiPL-HumanEval (PHP)
60
+ metrics:
61
+ - name: pass@1
62
+ type: pass@1
63
+ value: 24.2
64
+ verified: false
65
+ - task:
66
+ type: text-generation
67
+ dataset:
68
+ type: nuprl/MultiPL-E
69
+ name: MultiPL-HumanEval (Rust)
70
+ metrics:
71
+ - name: pass@1
72
+ type: pass@1
73
+ value: 23
74
+ verified: false
75
+ pipeline_tag: text-generation
76
+ ---
77
+
78
+ # QuantFactory/stable-code-instruct-3b-GGUF
79
+ This is quantized version of [stabilityai/stable-code-instruct-3b](https://huggingface.co/stabilityai/stable-code-instruct-3b) created using llama.cpp
80
+
81
+ # Model Description
82
+
83
+ [Try it out here: https://huggingface.co/spaces/stabilityai/stable-code-instruct-3b](https://huggingface.co/spaces/stabilityai/stable-code-instruct-3b)
84
+
85
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/63466107f7bd6326925fc770/O7ZkLgqoJprQEWAttX7Hj.png)
86
+
87
+
88
+
89
+ `stable-code-instruct-3b` is a 2.7B billion parameter decoder-only language model tuned from [`stable-code-3b`](https://huggingface.co/stabilityai/stable-code-3b/). This model was trained on a mix of publicly available datasets, synthetic datasets using [Direct Preference Optimization (DPO)](https://arxiv.org/abs/2305.18290).
90
+
91
+ This instruct tune demonstrates state-of-the-art performance (compared to models of similar size) on the MultiPL-E metrics across multiple programming languages tested using [BigCode's Evaluation Harness](https://github.com/bigcode-project/bigcode-evaluation-harness/tree/main), and on the code portions of
92
+ [MT Bench](https://klu.ai/glossary/mt-bench-eval).
93
+ The model is finetuned to make it useable in tasks like,
94
+ - General purpose Code/Software Engineering like conversations.
95
+ - SQL related generation and conversation.
96
+
97
+ Please note: For commercial use, please refer to https://stability.ai/license.
98
+
99
+ ## Usage
100
+ Here's how you can run the model use the model:
101
+
102
+ ```python
103
+
104
+ import torch
105
+ from transformers import AutoModelForCausalLM, AutoTokenizer
106
+ tokenizer = AutoTokenizer.from_pretrained("stabilityai/stable-code-instruct-3b", trust_remote_code=True)
107
+ model = AutoModelForCausalLM.from_pretrained("stabilityai/stable-code-instruct-3b", torch_dtype=torch.bfloat16, trust_remote_code=True)
108
+ model.eval()
109
+ model = model.cuda()
110
+
111
+ messages = [
112
+ {
113
+ "role": "system",
114
+ "content": "You are a helpful and polite assistant",
115
+ },
116
+ {
117
+ "role": "user",
118
+ "content": "Write a simple website in HTML. When a user clicks the button, it shows a random joke from a list of 4 jokes."
119
+ },
120
+ ]
121
+
122
+ prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
123
+
124
+ inputs = tokenizer([prompt], return_tensors="pt").to(model.device)
125
+
126
+ tokens = model.generate(
127
+ **inputs,
128
+ max_new_tokens=1024,
129
+ temperature=0.5,
130
+ top_p=0.95,
131
+ top_k=100,
132
+ do_sample=True,
133
+ use_cache=True
134
+ )
135
+
136
+ output = tokenizer.batch_decode(tokens[:, inputs.input_ids.shape[-1]:], skip_special_tokens=False)[0]
137
+ ```
138
+
139
+ ## Model Details
140
+
141
+ * **Developed by**: [Stability AI](https://stability.ai/)
142
+ * **Model type**: `Stable Code Instruct 3B` model is an auto-regressive language model based on the transformer decoder architecture.
143
+ * **Language(s)**: English
144
+ * **Paper**: [Stable Code Technical Report](https://drive.google.com/file/d/16-DGsR5-qwoPztZ6HcM7KSRUxIXrjlSm/view)
145
+ * **Library**: [Alignment Handbook](https://github.com/huggingface/alignment-handbook.git)
146
+ * **Finetuned from model**: [https://huggingface.co/stabilityai/stable-code-3b](https://huggingface.co/stabilityai/stable-code-3b)
147
+ * **License**: [StabilityAI Community License](https://huggingface.co/stabilityai/stable-code-instruct-3b/blob/main/LICENSE.md).
148
+ * **Commercial License**: to use this model commercially, please refer to https://stability.ai/license
149
+ * **Contact**: For questions and comments about the model, please email `lm@stability.ai`
150
+
151
+
152
+ ## Performance
153
+ ### Multi-PL Benchmark:
154
+ | Model | Size | Avg | Python | C++ | JavaScript | Java | PHP | Rust |
155
+ |------------------------------|------|------|--------|------|------------|------|------|------|
156
+ | Codellama Instruct | 7B | 0.30 | 0.33 | 0.31 | 0.31 | 0.29 | 0.31 | 0.25 |
157
+ | Deepseek Instruct | 1.3B | 0.44 | 0.52 | **0.52** | 0.41 | **0.46** | 0.45 | 0.28 |
158
+ | Stable Code Instruct (SFT) | 3B | 0.44 | 0.55 | 0.45 | 0.42 | 0.42 | 0.44 | 0.32 |
159
+ | Stable Code Instruct (DPO) | 3B | **0.47** | **0.59** | 0.49 | **0.49** | 0.44 | **0.45** | **0.37** |
160
+
161
+ ### MT-Bench Coding:
162
+ | Model | Size | Score |
163
+ |-----------------------------|------|-----------------|
164
+ | DeepSeek Coder | 1.3B | 4.6 |
165
+ | Stable Code Instruct (DPO) | 3B | **5.8**(ours) |
166
+ | Stable Code Instruct (SFT) | 3B | 5.5 |
167
+ | DeepSeek Coder | 6.7B | **6.9** |
168
+ | CodeLlama Instruct | 7B | 3.55 |
169
+ | StarChat2 | 15B | 5.7 |
170
+
171
+ ### SQL Performance
172
+ | Model | Size | Date | Group By | Order By | Ratio | Join | Where |
173
+ |-----------------------------|------|-------|----------|----------|-------|-------|-------|
174
+ | Stable Code Instruct (DPO) | 3B | 24.0% | 54.2% | 68.5% | 40.0% | 54.2% | 42.8% |
175
+ | DeepSeek-Coder Instruct | 1.3B | 24.0% | 37.1% | 51.4% | 34.3% | 45.7% | 45.7% |
176
+ | SQLCoder | 7B | 64.0% | 82.9% | 74.3% | 54.3% | 74.3% | 74.3% |
177
+
178
+
179
+
180
+
181
+ ## How to Cite Original Model
182
+ ```bibtex
183
+ @misc{stable-code-instruct-3b,
184
+ url={[https://huggingface.co/stabilityai/stable-code-3b](https://huggingface.co/stabilityai/stable-code-instruct-3b)},
185
+ title={Stable Code 3B},
186
+ author={Phung, Duy, and Pinnaparaju, Nikhil and Adithyan, Reshinth and Zhuravinskyi, Maksym and Tow, Jonathan and Cooper, Nathan}
187
+ }
188
+ ```
stable-code-instruct-3b.Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b8df215a6e25d25365209a60b42eba896a4d31d448273386301c62d71fe953c
3
+ size 1083756320
stable-code-instruct-3b.Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bcd7f114855117536df0eaab9a83e4463c627bf3781cb07b5f89a80e04f1ba0d
3
+ size 1508565280
stable-code-instruct-3b.Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3ef583154ac52437f0222378a31ab33c64fd1a8a34280e6f334131b7c2c3c49f
3
+ size 1391419680
stable-code-instruct-3b.Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:98948bfff724e21f078546894de0506c2509871b3aed839a54e3a18d540d1ff1
3
+ size 1254449440
stable-code-instruct-3b.Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e496e134a8e2caa4a62f2b8fb4aed239fc242352b9582e1148a6d09b0b3a7583
3
+ size 1608571680
stable-code-instruct-3b.Q4_1.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b4383b432730396051d6a2d1169953f97bb74e2b8563b998e4890989959ad941
3
+ size 1775217440
stable-code-instruct-3b.Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9c24979aec279ab3e86760997276a75a26ed2f4028ce532c24dbe81366dd94b1
3
+ size 1708596000
stable-code-instruct-3b.Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:587b8902fe02df445816871030372dfec3b8d0ad3338425b8d2e44c1b23825f8
3
+ size 1620695840
stable-code-instruct-3b.Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f51edde4067818ad953afa6b6a19d11cfa8588e00f3efa5259d03b80dd82e7d3
3
+ size 1941863200
stable-code-instruct-3b.Q5_1.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3c1159c5d8565956e66c683e13eab6475821de86b1f2bcdd4b48451e4a662246
3
+ size 2108508960
stable-code-instruct-3b.Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9dc8588b0ab0699afcff8dfaf3f3b04f416c74860f0b2e4ff2b8a8d9060179f4
3
+ size 1993390880
stable-code-instruct-3b.Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dc8a25e3ad74bfeb3568aaa41d10a4663ab8efc426a4762b9f0c2c4b0a88f12b
3
+ size 1941863200
stable-code-instruct-3b.Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:83a48ed74effcf636fdfb1ffa979cd7031ec89a10a6939f9623842d646eb12eb
3
+ size 2295985440
stable-code-instruct-3b.Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:44b6e8638245313f45b8dfe06850da007fd75de21c6cef392a1cb661402fb511
3
+ size 2972926240