Hugging Face's logo

MLA299
/
Test-SMALL-Model

Text Classification
Chinese
qwen3_5
Model card Files
xet
 Community
Jackie_Chen commited on
Commit
8e5371f
·
verified ·
1 Parent(s): 38d6c76

Upload folder using huggingface_hub

Browse files
Files changed (8) hide show
  1. .gitattributes +1 -0
  2. adapter_config.json +2127 -0
  3. adapters.safetensors +3 -0
  4. chat_template.jinja +158 -0
  5. config.json +98 -0
  6. processor_config.json +65 -0
  7. tokenizer.json +3 -0
  8. tokenizer_config.json +33 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
adapter_config.json ADDED
@@ -0,0 +1,2127 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "fine_tune_type": "lora",
3
+ "peft_type": "LORA",
4
+ "base_model_name_or_path": "unsloth/Qwen3.5-2B",
5
+ "learning_rate": 0.0002,
6
+ "max_steps": 4,
7
+ "max_seq_length": 2048,
8
+ "use_cce": true,
9
+ "base_quantization_config": {
10
+ "language_model.model.layers.0.linear_attn.in_proj_qkv": {
11
+ "group_size": 64,
12
+ "bits": 4,
13
+ "mode": "affine"
14
+ },
15
+ "language_model.model.layers.0.linear_attn.in_proj_z": {
16
+ "group_size": 64,
17
+ "bits": 4,
18
+ "mode": "affine"
19
+ },
20
+ "language_model.model.layers.0.linear_attn.in_proj_b": {
21
+ "group_size": 64,
22
+ "bits": 4,
23
+ "mode": "affine"
24
+ },
25
+ "language_model.model.layers.0.linear_attn.in_proj_a": {
26
+ "group_size": 64,
27
+ "bits": 4,
28
+ "mode": "affine"
29
+ },
30
+ "language_model.model.layers.0.linear_attn.out_proj": {
31
+ "group_size": 64,
32
+ "bits": 4,
33
+ "mode": "affine"
34
+ },
35
+ "language_model.model.layers.0.mlp.gate_proj": {
36
+ "group_size": 64,
37
+ "bits": 4,
38
+ "mode": "affine"
39
+ },
40
+ "language_model.model.layers.0.mlp.down_proj": {
41
+ "group_size": 64,
42
+ "bits": 4,
43
+ "mode": "affine"
44
+ },
45
+ "language_model.model.layers.0.mlp.up_proj": {
46
+ "group_size": 64,
47
+ "bits": 4,
48
+ "mode": "affine"
49
+ },
50
+ "language_model.model.layers.1.linear_attn.in_proj_qkv": {
51
+ "group_size": 64,
52
+ "bits": 4,
53
+ "mode": "affine"
54
+ },
55
+ "language_model.model.layers.1.linear_attn.in_proj_z": {
56
+ "group_size": 64,
57
+ "bits": 4,
58
+ "mode": "affine"
59
+ },
60
+ "language_model.model.layers.1.linear_attn.in_proj_b": {
61
+ "group_size": 64,
62
+ "bits": 4,
63
+ "mode": "affine"
64
+ },
65
+ "language_model.model.layers.1.linear_attn.in_proj_a": {
66
+ "group_size": 64,
67
+ "bits": 4,
68
+ "mode": "affine"
69
+ },
70
+ "language_model.model.layers.1.linear_attn.out_proj": {
71
+ "group_size": 64,
72
+ "bits": 4,
73
+ "mode": "affine"
74
+ },
75
+ "language_model.model.layers.1.mlp.gate_proj": {
76
+ "group_size": 64,
77
+ "bits": 4,
78
+ "mode": "affine"
79
+ },
80
+ "language_model.model.layers.1.mlp.down_proj": {
81
+ "group_size": 64,
82
+ "bits": 4,
83
+ "mode": "affine"
84
+ },
85
+ "language_model.model.layers.1.mlp.up_proj": {
86
+ "group_size": 64,
87
+ "bits": 4,
88
+ "mode": "affine"
89
+ },
90
+ "language_model.model.layers.2.linear_attn.in_proj_qkv": {
91
+ "group_size": 64,
92
+ "bits": 4,
93
+ "mode": "affine"
94
+ },
95
+ "language_model.model.layers.2.linear_attn.in_proj_z": {
96
+ "group_size": 64,
97
+ "bits": 4,
98
+ "mode": "affine"
99
+ },
100
+ "language_model.model.layers.2.linear_attn.in_proj_b": {
101
+ "group_size": 64,
102
+ "bits": 4,
103
+ "mode": "affine"
104
+ },
105
+ "language_model.model.layers.2.linear_attn.in_proj_a": {
106
+ "group_size": 64,
107
+ "bits": 4,
108
+ "mode": "affine"
109
+ },
110
+ "language_model.model.layers.2.linear_attn.out_proj": {
111
+ "group_size": 64,
112
+ "bits": 4,
113
+ "mode": "affine"
114
+ },
115
+ "language_model.model.layers.2.mlp.gate_proj": {
116
+ "group_size": 64,
117
+ "bits": 4,
118
+ "mode": "affine"
119
+ },
120
+ "language_model.model.layers.2.mlp.down_proj": {
121
+ "group_size": 64,
122
+ "bits": 4,
123
+ "mode": "affine"
124
+ },
125
+ "language_model.model.layers.2.mlp.up_proj": {
126
+ "group_size": 64,
127
+ "bits": 4,
128
+ "mode": "affine"
129
+ },
130
+ "language_model.model.layers.3.self_attn.q_proj": {
131
+ "group_size": 64,
132
+ "bits": 4,
133
+ "mode": "affine"
134
+ },
135
+ "language_model.model.layers.3.self_attn.k_proj": {
136
+ "group_size": 64,
137
+ "bits": 4,
138
+ "mode": "affine"
139
+ },
140
+ "language_model.model.layers.3.self_attn.v_proj": {
141
+ "group_size": 64,
142
+ "bits": 4,
143
+ "mode": "affine"
144
+ },
145
+ "language_model.model.layers.3.self_attn.o_proj": {
146
+ "group_size": 64,
147
+ "bits": 4,
148
+ "mode": "affine"
149
+ },
150
+ "language_model.model.layers.3.mlp.gate_proj": {
151
+ "group_size": 64,
152
+ "bits": 4,
153
+ "mode": "affine"
154
+ },
155
+ "language_model.model.layers.3.mlp.down_proj": {
156
+ "group_size": 64,
157
+ "bits": 4,
158
+ "mode": "affine"
159
+ },
160
+ "language_model.model.layers.3.mlp.up_proj": {
161
+ "group_size": 64,
162
+ "bits": 4,
163
+ "mode": "affine"
164
+ },
165
+ "language_model.model.layers.4.linear_attn.in_proj_qkv": {
166
+ "group_size": 64,
167
+ "bits": 4,
168
+ "mode": "affine"
169
+ },
170
+ "language_model.model.layers.4.linear_attn.in_proj_z": {
171
+ "group_size": 64,
172
+ "bits": 4,
173
+ "mode": "affine"
174
+ },
175
+ "language_model.model.layers.4.linear_attn.in_proj_b": {
176
+ "group_size": 64,
177
+ "bits": 4,
178
+ "mode": "affine"
179
+ },
180
+ "language_model.model.layers.4.linear_attn.in_proj_a": {
181
+ "group_size": 64,
182
+ "bits": 4,
183
+ "mode": "affine"
184
+ },
185
+ "language_model.model.layers.4.linear_attn.out_proj": {
186
+ "group_size": 64,
187
+ "bits": 4,
188
+ "mode": "affine"
189
+ },
190
+ "language_model.model.layers.4.mlp.gate_proj": {
191
+ "group_size": 64,
192
+ "bits": 4,
193
+ "mode": "affine"
194
+ },
195
+ "language_model.model.layers.4.mlp.down_proj": {
196
+ "group_size": 64,
197
+ "bits": 4,
198
+ "mode": "affine"
199
+ },
200
+ "language_model.model.layers.4.mlp.up_proj": {
201
+ "group_size": 64,
202
+ "bits": 4,
203
+ "mode": "affine"
204
+ },
205
+ "language_model.model.layers.5.linear_attn.in_proj_qkv": {
206
+ "group_size": 64,
207
+ "bits": 4,
208
+ "mode": "affine"
209
+ },
210
+ "language_model.model.layers.5.linear_attn.in_proj_z": {
211
+ "group_size": 64,
212
+ "bits": 4,
213
+ "mode": "affine"
214
+ },
215
+ "language_model.model.layers.5.linear_attn.in_proj_b": {
216
+ "group_size": 64,
217
+ "bits": 4,
218
+ "mode": "affine"
219
+ },
220
+ "language_model.model.layers.5.linear_attn.in_proj_a": {
221
+ "group_size": 64,
222
+ "bits": 4,
223
+ "mode": "affine"
224
+ },
225
+ "language_model.model.layers.5.linear_attn.out_proj": {
226
+ "group_size": 64,
227
+ "bits": 4,
228
+ "mode": "affine"
229
+ },
230
+ "language_model.model.layers.5.mlp.gate_proj": {
231
+ "group_size": 64,
232
+ "bits": 4,
233
+ "mode": "affine"
234
+ },
235
+ "language_model.model.layers.5.mlp.down_proj": {
236
+ "group_size": 64,
237
+ "bits": 4,
238
+ "mode": "affine"
239
+ },
240
+ "language_model.model.layers.5.mlp.up_proj": {
241
+ "group_size": 64,
242
+ "bits": 4,
243
+ "mode": "affine"
244
+ },
245
+ "language_model.model.layers.6.linear_attn.in_proj_qkv": {
246
+ "group_size": 64,
247
+ "bits": 4,
248
+ "mode": "affine"
249
+ },
250
+ "language_model.model.layers.6.linear_attn.in_proj_z": {
251
+ "group_size": 64,
252
+ "bits": 4,
253
+ "mode": "affine"
254
+ },
255
+ "language_model.model.layers.6.linear_attn.in_proj_b": {
256
+ "group_size": 64,
257
+ "bits": 4,
258
+ "mode": "affine"
259
+ },
260
+ "language_model.model.layers.6.linear_attn.in_proj_a": {
261
+ "group_size": 64,
262
+ "bits": 4,
263
+ "mode": "affine"
264
+ },
265
+ "language_model.model.layers.6.linear_attn.out_proj": {
266
+ "group_size": 64,
267
+ "bits": 4,
268
+ "mode": "affine"
269
+ },
270
+ "language_model.model.layers.6.mlp.gate_proj": {
271
+ "group_size": 64,
272
+ "bits": 4,
273
+ "mode": "affine"
274
+ },
275
+ "language_model.model.layers.6.mlp.down_proj": {
276
+ "group_size": 64,
277
+ "bits": 4,
278
+ "mode": "affine"
279
+ },
280
+ "language_model.model.layers.6.mlp.up_proj": {
281
+ "group_size": 64,
282
+ "bits": 4,
283
+ "mode": "affine"
284
+ },
285
+ "language_model.model.layers.7.self_attn.q_proj": {
286
+ "group_size": 64,
287
+ "bits": 4,
288
+ "mode": "affine"
289
+ },
290
+ "language_model.model.layers.7.self_attn.k_proj": {
291
+ "group_size": 64,
292
+ "bits": 4,
293
+ "mode": "affine"
294
+ },
295
+ "language_model.model.layers.7.self_attn.v_proj": {
296
+ "group_size": 64,
297
+ "bits": 4,
298
+ "mode": "affine"
299
+ },
300
+ "language_model.model.layers.7.self_attn.o_proj": {
301
+ "group_size": 64,
302
+ "bits": 4,
303
+ "mode": "affine"
304
+ },
305
+ "language_model.model.layers.7.mlp.gate_proj": {
306
+ "group_size": 64,
307
+ "bits": 4,
308
+ "mode": "affine"
309
+ },
310
+ "language_model.model.layers.7.mlp.down_proj": {
311
+ "group_size": 64,
312
+ "bits": 4,
313
+ "mode": "affine"
314
+ },
315
+ "language_model.model.layers.7.mlp.up_proj": {
316
+ "group_size": 64,
317
+ "bits": 4,
318
+ "mode": "affine"
319
+ },
320
+ "language_model.model.layers.8.linear_attn.in_proj_qkv": {
321
+ "group_size": 64,
322
+ "bits": 4,
323
+ "mode": "affine"
324
+ },
325
+ "language_model.model.layers.8.linear_attn.in_proj_z": {
326
+ "group_size": 64,
327
+ "bits": 4,
328
+ "mode": "affine"
329
+ },
330
+ "language_model.model.layers.8.linear_attn.in_proj_b": {
331
+ "group_size": 64,
332
+ "bits": 4,
333
+ "mode": "affine"
334
+ },
335
+ "language_model.model.layers.8.linear_attn.in_proj_a": {
336
+ "group_size": 64,
337
+ "bits": 4,
338
+ "mode": "affine"
339
+ },
340
+ "language_model.model.layers.8.linear_attn.out_proj": {
341
+ "group_size": 64,
342
+ "bits": 4,
343
+ "mode": "affine"
344
+ },
345
+ "language_model.model.layers.8.mlp.gate_proj": {
346
+ "group_size": 64,
347
+ "bits": 4,
348
+ "mode": "affine"
349
+ },
350
+ "language_model.model.layers.8.mlp.down_proj": {
351
+ "group_size": 64,
352
+ "bits": 4,
353
+ "mode": "affine"
354
+ },
355
+ "language_model.model.layers.8.mlp.up_proj": {
356
+ "group_size": 64,
357
+ "bits": 4,
358
+ "mode": "affine"
359
+ },
360
+ "language_model.model.layers.9.linear_attn.in_proj_qkv": {
361
+ "group_size": 64,
362
+ "bits": 4,
363
+ "mode": "affine"
364
+ },
365
+ "language_model.model.layers.9.linear_attn.in_proj_z": {
366
+ "group_size": 64,
367
+ "bits": 4,
368
+ "mode": "affine"
369
+ },
370
+ "language_model.model.layers.9.linear_attn.in_proj_b": {
371
+ "group_size": 64,
372
+ "bits": 4,
373
+ "mode": "affine"
374
+ },
375
+ "language_model.model.layers.9.linear_attn.in_proj_a": {
376
+ "group_size": 64,
377
+ "bits": 4,
378
+ "mode": "affine"
379
+ },
380
+ "language_model.model.layers.9.linear_attn.out_proj": {
381
+ "group_size": 64,
382
+ "bits": 4,
383
+ "mode": "affine"
384
+ },
385
+ "language_model.model.layers.9.mlp.gate_proj": {
386
+ "group_size": 64,
387
+ "bits": 4,
388
+ "mode": "affine"
389
+ },
390
+ "language_model.model.layers.9.mlp.down_proj": {
391
+ "group_size": 64,
392
+ "bits": 4,
393
+ "mode": "affine"
394
+ },
395
+ "language_model.model.layers.9.mlp.up_proj": {
396
+ "group_size": 64,
397
+ "bits": 4,
398
+ "mode": "affine"
399
+ },
400
+ "language_model.model.layers.10.linear_attn.in_proj_qkv": {
401
+ "group_size": 64,
402
+ "bits": 4,
403
+ "mode": "affine"
404
+ },
405
+ "language_model.model.layers.10.linear_attn.in_proj_z": {
406
+ "group_size": 64,
407
+ "bits": 4,
408
+ "mode": "affine"
409
+ },
410
+ "language_model.model.layers.10.linear_attn.in_proj_b": {
411
+ "group_size": 64,
412
+ "bits": 4,
413
+ "mode": "affine"
414
+ },
415
+ "language_model.model.layers.10.linear_attn.in_proj_a": {
416
+ "group_size": 64,
417
+ "bits": 4,
418
+ "mode": "affine"
419
+ },
420
+ "language_model.model.layers.10.linear_attn.out_proj": {
421
+ "group_size": 64,
422
+ "bits": 4,
423
+ "mode": "affine"
424
+ },
425
+ "language_model.model.layers.10.mlp.gate_proj": {
426
+ "group_size": 64,
427
+ "bits": 4,
428
+ "mode": "affine"
429
+ },
430
+ "language_model.model.layers.10.mlp.down_proj": {
431
+ "group_size": 64,
432
+ "bits": 4,
433
+ "mode": "affine"
434
+ },
435
+ "language_model.model.layers.10.mlp.up_proj": {
436
+ "group_size": 64,
437
+ "bits": 4,
438
+ "mode": "affine"
439
+ },
440
+ "language_model.model.layers.11.self_attn.q_proj": {
441
+ "group_size": 64,
442
+ "bits": 4,
443
+ "mode": "affine"
444
+ },
445
+ "language_model.model.layers.11.self_attn.k_proj": {
446
+ "group_size": 64,
447
+ "bits": 4,
448
+ "mode": "affine"
449
+ },
450
+ "language_model.model.layers.11.self_attn.v_proj": {
451
+ "group_size": 64,
452
+ "bits": 4,
453
+ "mode": "affine"
454
+ },
455
+ "language_model.model.layers.11.self_attn.o_proj": {
456
+ "group_size": 64,
457
+ "bits": 4,
458
+ "mode": "affine"
459
+ },
460
+ "language_model.model.layers.11.mlp.gate_proj": {
461
+ "group_size": 64,
462
+ "bits": 4,
463
+ "mode": "affine"
464
+ },
465
+ "language_model.model.layers.11.mlp.down_proj": {
466
+ "group_size": 64,
467
+ "bits": 4,
468
+ "mode": "affine"
469
+ },
470
+ "language_model.model.layers.11.mlp.up_proj": {
471
+ "group_size": 64,
472
+ "bits": 4,
473
+ "mode": "affine"
474
+ },
475
+ "language_model.model.layers.12.linear_attn.in_proj_qkv": {
476
+ "group_size": 64,
477
+ "bits": 4,
478
+ "mode": "affine"
479
+ },
480
+ "language_model.model.layers.12.linear_attn.in_proj_z": {
481
+ "group_size": 64,
482
+ "bits": 4,
483
+ "mode": "affine"
484
+ },
485
+ "language_model.model.layers.12.linear_attn.in_proj_b": {
486
+ "group_size": 64,
487
+ "bits": 4,
488
+ "mode": "affine"
489
+ },
490
+ "language_model.model.layers.12.linear_attn.in_proj_a": {
491
+ "group_size": 64,
492
+ "bits": 4,
493
+ "mode": "affine"
494
+ },
495
+ "language_model.model.layers.12.linear_attn.out_proj": {
496
+ "group_size": 64,
497
+ "bits": 4,
498
+ "mode": "affine"
499
+ },
500
+ "language_model.model.layers.12.mlp.gate_proj": {
501
+ "group_size": 64,
502
+ "bits": 4,
503
+ "mode": "affine"
504
+ },
505
+ "language_model.model.layers.12.mlp.down_proj": {
506
+ "group_size": 64,
507
+ "bits": 4,
508
+ "mode": "affine"
509
+ },
510
+ "language_model.model.layers.12.mlp.up_proj": {
511
+ "group_size": 64,
512
+ "bits": 4,
513
+ "mode": "affine"
514
+ },
515
+ "language_model.model.layers.13.linear_attn.in_proj_qkv": {
516
+ "group_size": 64,
517
+ "bits": 4,
518
+ "mode": "affine"
519
+ },
520
+ "language_model.model.layers.13.linear_attn.in_proj_z": {
521
+ "group_size": 64,
522
+ "bits": 4,
523
+ "mode": "affine"
524
+ },
525
+ "language_model.model.layers.13.linear_attn.in_proj_b": {
526
+ "group_size": 64,
527
+ "bits": 4,
528
+ "mode": "affine"
529
+ },
530
+ "language_model.model.layers.13.linear_attn.in_proj_a": {
531
+ "group_size": 64,
532
+ "bits": 4,
533
+ "mode": "affine"
534
+ },
535
+ "language_model.model.layers.13.linear_attn.out_proj": {
536
+ "group_size": 64,
537
+ "bits": 4,
538
+ "mode": "affine"
539
+ },
540
+ "language_model.model.layers.13.mlp.gate_proj": {
541
+ "group_size": 64,
542
+ "bits": 4,
543
+ "mode": "affine"
544
+ },
545
+ "language_model.model.layers.13.mlp.down_proj": {
546
+ "group_size": 64,
547
+ "bits": 4,
548
+ "mode": "affine"
549
+ },
550
+ "language_model.model.layers.13.mlp.up_proj": {
551
+ "group_size": 64,
552
+ "bits": 4,
553
+ "mode": "affine"
554
+ },
555
+ "language_model.model.layers.14.linear_attn.in_proj_qkv": {
556
+ "group_size": 64,
557
+ "bits": 4,
558
+ "mode": "affine"
559
+ },
560
+ "language_model.model.layers.14.linear_attn.in_proj_z": {
561
+ "group_size": 64,
562
+ "bits": 4,
563
+ "mode": "affine"
564
+ },
565
+ "language_model.model.layers.14.linear_attn.in_proj_b": {
566
+ "group_size": 64,
567
+ "bits": 4,
568
+ "mode": "affine"
569
+ },
570
+ "language_model.model.layers.14.linear_attn.in_proj_a": {
571
+ "group_size": 64,
572
+ "bits": 4,
573
+ "mode": "affine"
574
+ },
575
+ "language_model.model.layers.14.linear_attn.out_proj": {
576
+ "group_size": 64,
577
+ "bits": 4,
578
+ "mode": "affine"
579
+ },
580
+ "language_model.model.layers.14.mlp.gate_proj": {
581
+ "group_size": 64,
582
+ "bits": 4,
583
+ "mode": "affine"
584
+ },
585
+ "language_model.model.layers.14.mlp.down_proj": {
586
+ "group_size": 64,
587
+ "bits": 4,
588
+ "mode": "affine"
589
+ },
590
+ "language_model.model.layers.14.mlp.up_proj": {
591
+ "group_size": 64,
592
+ "bits": 4,
593
+ "mode": "affine"
594
+ },
595
+ "language_model.model.layers.15.self_attn.q_proj": {
596
+ "group_size": 64,
597
+ "bits": 4,
598
+ "mode": "affine"
599
+ },
600
+ "language_model.model.layers.15.self_attn.k_proj": {
601
+ "group_size": 64,
602
+ "bits": 4,
603
+ "mode": "affine"
604
+ },
605
+ "language_model.model.layers.15.self_attn.v_proj": {
606
+ "group_size": 64,
607
+ "bits": 4,
608
+ "mode": "affine"
609
+ },
610
+ "language_model.model.layers.15.self_attn.o_proj": {
611
+ "group_size": 64,
612
+ "bits": 4,
613
+ "mode": "affine"
614
+ },
615
+ "language_model.model.layers.15.mlp.gate_proj": {
616
+ "group_size": 64,
617
+ "bits": 4,
618
+ "mode": "affine"
619
+ },
620
+ "language_model.model.layers.15.mlp.down_proj": {
621
+ "group_size": 64,
622
+ "bits": 4,
623
+ "mode": "affine"
624
+ },
625
+ "language_model.model.layers.15.mlp.up_proj": {
626
+ "group_size": 64,
627
+ "bits": 4,
628
+ "mode": "affine"
629
+ },
630
+ "language_model.model.layers.16.linear_attn.in_proj_qkv": {
631
+ "group_size": 64,
632
+ "bits": 4,
633
+ "mode": "affine"
634
+ },
635
+ "language_model.model.layers.16.linear_attn.in_proj_z": {
636
+ "group_size": 64,
637
+ "bits": 4,
638
+ "mode": "affine"
639
+ },
640
+ "language_model.model.layers.16.linear_attn.in_proj_b": {
641
+ "group_size": 64,
642
+ "bits": 4,
643
+ "mode": "affine"
644
+ },
645
+ "language_model.model.layers.16.linear_attn.in_proj_a": {
646
+ "group_size": 64,
647
+ "bits": 4,
648
+ "mode": "affine"
649
+ },
650
+ "language_model.model.layers.16.linear_attn.out_proj": {
651
+ "group_size": 64,
652
+ "bits": 4,
653
+ "mode": "affine"
654
+ },
655
+ "language_model.model.layers.16.mlp.gate_proj": {
656
+ "group_size": 64,
657
+ "bits": 4,
658
+ "mode": "affine"
659
+ },
660
+ "language_model.model.layers.16.mlp.down_proj": {
661
+ "group_size": 64,
662
+ "bits": 4,
663
+ "mode": "affine"
664
+ },
665
+ "language_model.model.layers.16.mlp.up_proj": {
666
+ "group_size": 64,
667
+ "bits": 4,
668
+ "mode": "affine"
669
+ },
670
+ "language_model.model.layers.17.linear_attn.in_proj_qkv": {
671
+ "group_size": 64,
672
+ "bits": 4,
673
+ "mode": "affine"
674
+ },
675
+ "language_model.model.layers.17.linear_attn.in_proj_z": {
676
+ "group_size": 64,
677
+ "bits": 4,
678
+ "mode": "affine"
679
+ },
680
+ "language_model.model.layers.17.linear_attn.in_proj_b": {
681
+ "group_size": 64,
682
+ "bits": 4,
683
+ "mode": "affine"
684
+ },
685
+ "language_model.model.layers.17.linear_attn.in_proj_a": {
686
+ "group_size": 64,
687
+ "bits": 4,
688
+ "mode": "affine"
689
+ },
690
+ "language_model.model.layers.17.linear_attn.out_proj": {
691
+ "group_size": 64,
692
+ "bits": 4,
693
+ "mode": "affine"
694
+ },
695
+ "language_model.model.layers.17.mlp.gate_proj": {
696
+ "group_size": 64,
697
+ "bits": 4,
698
+ "mode": "affine"
699
+ },
700
+ "language_model.model.layers.17.mlp.down_proj": {
701
+ "group_size": 64,
702
+ "bits": 4,
703
+ "mode": "affine"
704
+ },
705
+ "language_model.model.layers.17.mlp.up_proj": {
706
+ "group_size": 64,
707
+ "bits": 4,
708
+ "mode": "affine"
709
+ },
710
+ "language_model.model.layers.18.linear_attn.in_proj_qkv": {
711
+ "group_size": 64,
712
+ "bits": 4,
713
+ "mode": "affine"
714
+ },
715
+ "language_model.model.layers.18.linear_attn.in_proj_z": {
716
+ "group_size": 64,
717
+ "bits": 4,
718
+ "mode": "affine"
719
+ },
720
+ "language_model.model.layers.18.linear_attn.in_proj_b": {
721
+ "group_size": 64,
722
+ "bits": 4,
723
+ "mode": "affine"
724
+ },
725
+ "language_model.model.layers.18.linear_attn.in_proj_a": {
726
+ "group_size": 64,
727
+ "bits": 4,
728
+ "mode": "affine"
729
+ },
730
+ "language_model.model.layers.18.linear_attn.out_proj": {
731
+ "group_size": 64,
732
+ "bits": 4,
733
+ "mode": "affine"
734
+ },
735
+ "language_model.model.layers.18.mlp.gate_proj": {
736
+ "group_size": 64,
737
+ "bits": 4,
738
+ "mode": "affine"
739
+ },
740
+ "language_model.model.layers.18.mlp.down_proj": {
741
+ "group_size": 64,
742
+ "bits": 4,
743
+ "mode": "affine"
744
+ },
745
+ "language_model.model.layers.18.mlp.up_proj": {
746
+ "group_size": 64,
747
+ "bits": 4,
748
+ "mode": "affine"
749
+ },
750
+ "language_model.model.layers.19.self_attn.q_proj": {
751
+ "group_size": 64,
752
+ "bits": 4,
753
+ "mode": "affine"
754
+ },
755
+ "language_model.model.layers.19.self_attn.k_proj": {
756
+ "group_size": 64,
757
+ "bits": 4,
758
+ "mode": "affine"
759
+ },
760
+ "language_model.model.layers.19.self_attn.v_proj": {
761
+ "group_size": 64,
762
+ "bits": 4,
763
+ "mode": "affine"
764
+ },
765
+ "language_model.model.layers.19.self_attn.o_proj": {
766
+ "group_size": 64,
767
+ "bits": 4,
768
+ "mode": "affine"
769
+ },
770
+ "language_model.model.layers.19.mlp.gate_proj": {
771
+ "group_size": 64,
772
+ "bits": 4,
773
+ "mode": "affine"
774
+ },
775
+ "language_model.model.layers.19.mlp.down_proj": {
776
+ "group_size": 64,
777
+ "bits": 4,
778
+ "mode": "affine"
779
+ },
780
+ "language_model.model.layers.19.mlp.up_proj": {
781
+ "group_size": 64,
782
+ "bits": 4,
783
+ "mode": "affine"
784
+ },
785
+ "language_model.model.layers.20.linear_attn.in_proj_qkv": {
786
+ "group_size": 64,
787
+ "bits": 4,
788
+ "mode": "affine"
789
+ },
790
+ "language_model.model.layers.20.linear_attn.in_proj_z": {
791
+ "group_size": 64,
792
+ "bits": 4,
793
+ "mode": "affine"
794
+ },
795
+ "language_model.model.layers.20.linear_attn.in_proj_b": {
796
+ "group_size": 64,
797
+ "bits": 4,
798
+ "mode": "affine"
799
+ },
800
+ "language_model.model.layers.20.linear_attn.in_proj_a": {
801
+ "group_size": 64,
802
+ "bits": 4,
803
+ "mode": "affine"
804
+ },
805
+ "language_model.model.layers.20.linear_attn.out_proj": {
806
+ "group_size": 64,
807
+ "bits": 4,
808
+ "mode": "affine"
809
+ },
810
+ "language_model.model.layers.20.mlp.gate_proj": {
811
+ "group_size": 64,
812
+ "bits": 4,
813
+ "mode": "affine"
814
+ },
815
+ "language_model.model.layers.20.mlp.down_proj": {
816
+ "group_size": 64,
817
+ "bits": 4,
818
+ "mode": "affine"
819
+ },
820
+ "language_model.model.layers.20.mlp.up_proj": {
821
+ "group_size": 64,
822
+ "bits": 4,
823
+ "mode": "affine"
824
+ },
825
+ "language_model.model.layers.21.linear_attn.in_proj_qkv": {
826
+ "group_size": 64,
827
+ "bits": 4,
828
+ "mode": "affine"
829
+ },
830
+ "language_model.model.layers.21.linear_attn.in_proj_z": {
831
+ "group_size": 64,
832
+ "bits": 4,
833
+ "mode": "affine"
834
+ },
835
+ "language_model.model.layers.21.linear_attn.in_proj_b": {
836
+ "group_size": 64,
837
+ "bits": 4,
838
+ "mode": "affine"
839
+ },
840
+ "language_model.model.layers.21.linear_attn.in_proj_a": {
841
+ "group_size": 64,
842
+ "bits": 4,
843
+ "mode": "affine"
844
+ },
845
+ "language_model.model.layers.21.linear_attn.out_proj": {
846
+ "group_size": 64,
847
+ "bits": 4,
848
+ "mode": "affine"
849
+ },
850
+ "language_model.model.layers.21.mlp.gate_proj": {
851
+ "group_size": 64,
852
+ "bits": 4,
853
+ "mode": "affine"
854
+ },
855
+ "language_model.model.layers.21.mlp.down_proj": {
856
+ "group_size": 64,
857
+ "bits": 4,
858
+ "mode": "affine"
859
+ },
860
+ "language_model.model.layers.21.mlp.up_proj": {
861
+ "group_size": 64,
862
+ "bits": 4,
863
+ "mode": "affine"
864
+ },
865
+ "language_model.model.layers.22.linear_attn.in_proj_qkv": {
866
+ "group_size": 64,
867
+ "bits": 4,
868
+ "mode": "affine"
869
+ },
870
+ "language_model.model.layers.22.linear_attn.in_proj_z": {
871
+ "group_size": 64,
872
+ "bits": 4,
873
+ "mode": "affine"
874
+ },
875
+ "language_model.model.layers.22.linear_attn.in_proj_b": {
876
+ "group_size": 64,
877
+ "bits": 4,
878
+ "mode": "affine"
879
+ },
880
+ "language_model.model.layers.22.linear_attn.in_proj_a": {
881
+ "group_size": 64,
882
+ "bits": 4,
883
+ "mode": "affine"
884
+ },
885
+ "language_model.model.layers.22.linear_attn.out_proj": {
886
+ "group_size": 64,
887
+ "bits": 4,
888
+ "mode": "affine"
889
+ },
890
+ "language_model.model.layers.22.mlp.gate_proj": {
891
+ "group_size": 64,
892
+ "bits": 4,
893
+ "mode": "affine"
894
+ },
895
+ "language_model.model.layers.22.mlp.down_proj": {
896
+ "group_size": 64,
897
+ "bits": 4,
898
+ "mode": "affine"
899
+ },
900
+ "language_model.model.layers.22.mlp.up_proj": {
901
+ "group_size": 64,
902
+ "bits": 4,
903
+ "mode": "affine"
904
+ },
905
+ "language_model.model.layers.23.self_attn.q_proj": {
906
+ "group_size": 64,
907
+ "bits": 4,
908
+ "mode": "affine"
909
+ },
910
+ "language_model.model.layers.23.self_attn.k_proj": {
911
+ "group_size": 64,
912
+ "bits": 4,
913
+ "mode": "affine"
914
+ },
915
+ "language_model.model.layers.23.self_attn.v_proj": {
916
+ "group_size": 64,
917
+ "bits": 4,
918
+ "mode": "affine"
919
+ },
920
+ "language_model.model.layers.23.self_attn.o_proj": {
921
+ "group_size": 64,
922
+ "bits": 4,
923
+ "mode": "affine"
924
+ },
925
+ "language_model.model.layers.23.mlp.gate_proj": {
926
+ "group_size": 64,
927
+ "bits": 4,
928
+ "mode": "affine"
929
+ },
930
+ "language_model.model.layers.23.mlp.down_proj": {
931
+ "group_size": 64,
932
+ "bits": 4,
933
+ "mode": "affine"
934
+ },
935
+ "language_model.model.layers.23.mlp.up_proj": {
936
+ "group_size": 64,
937
+ "bits": 4,
938
+ "mode": "affine"
939
+ }
940
+ },
941
+ "base_quantization_policy": {
942
+ "enabled": true,
943
+ "bits": 4,
944
+ "group_size": 64,
945
+ "mode": "affine",
946
+ "source": "load_in_4bit",
947
+ "quantize_modules": null,
948
+ "has_callable_predicate": false,
949
+ "force_requantize": false
950
+ },
951
+ "base_quantized_source": "runtime",
952
+ "num_layers": 24,
953
+ "lora_parameters": {
954
+ "rank": 16,
955
+ "scale": 4.0,
956
+ "dropout": 0.0,
957
+ "keys": [
958
+ "language_model.model.layers.23.mlp.up_proj",
959
+ "language_model.model.layers.23.mlp.down_proj",
960
+ "language_model.model.layers.23.mlp.gate_proj",
961
+ "language_model.model.layers.23.self_attn.o_proj",
962
+ "language_model.model.layers.23.self_attn.v_proj",
963
+ "language_model.model.layers.23.self_attn.k_proj",
964
+ "language_model.model.layers.23.self_attn.q_proj",
965
+ "language_model.model.layers.22.mlp.up_proj",
966
+ "language_model.model.layers.22.mlp.down_proj",
967
+ "language_model.model.layers.22.mlp.gate_proj",
968
+ "language_model.model.layers.22.linear_attn.out_proj",
969
+ "language_model.model.layers.21.mlp.up_proj",
970
+ "language_model.model.layers.21.mlp.down_proj",
971
+ "language_model.model.layers.21.mlp.gate_proj",
972
+ "language_model.model.layers.21.linear_attn.out_proj",
973
+ "language_model.model.layers.20.mlp.up_proj",
974
+ "language_model.model.layers.20.mlp.down_proj",
975
+ "language_model.model.layers.20.mlp.gate_proj",
976
+ "language_model.model.layers.20.linear_attn.out_proj",
977
+ "language_model.model.layers.19.mlp.up_proj",
978
+ "language_model.model.layers.19.mlp.down_proj",
979
+ "language_model.model.layers.19.mlp.gate_proj",
980
+ "language_model.model.layers.19.self_attn.o_proj",
981
+ "language_model.model.layers.19.self_attn.v_proj",
982
+ "language_model.model.layers.19.self_attn.k_proj",
983
+ "language_model.model.layers.19.self_attn.q_proj",
984
+ "language_model.model.layers.18.mlp.up_proj",
985
+ "language_model.model.layers.18.mlp.down_proj",
986
+ "language_model.model.layers.18.mlp.gate_proj",
987
+ "language_model.model.layers.18.linear_attn.out_proj",
988
+ "language_model.model.layers.17.mlp.up_proj",
989
+ "language_model.model.layers.17.mlp.down_proj",
990
+ "language_model.model.layers.17.mlp.gate_proj",
991
+ "language_model.model.layers.17.linear_attn.out_proj",
992
+ "language_model.model.layers.16.mlp.up_proj",
993
+ "language_model.model.layers.16.mlp.down_proj",
994
+ "language_model.model.layers.16.mlp.gate_proj",
995
+ "language_model.model.layers.16.linear_attn.out_proj",
996
+ "language_model.model.layers.15.mlp.up_proj",
997
+ "language_model.model.layers.15.mlp.down_proj",
998
+ "language_model.model.layers.15.mlp.gate_proj",
999
+ "language_model.model.layers.15.self_attn.o_proj",
1000
+ "language_model.model.layers.15.self_attn.v_proj",
1001
+ "language_model.model.layers.15.self_attn.k_proj",
1002
+ "language_model.model.layers.15.self_attn.q_proj",
1003
+ "language_model.model.layers.14.mlp.up_proj",
1004
+ "language_model.model.layers.14.mlp.down_proj",
1005
+ "language_model.model.layers.14.mlp.gate_proj",
1006
+ "language_model.model.layers.14.linear_attn.out_proj",
1007
+ "language_model.model.layers.13.mlp.up_proj",
1008
+ "language_model.model.layers.13.mlp.down_proj",
1009
+ "language_model.model.layers.13.mlp.gate_proj",
1010
+ "language_model.model.layers.13.linear_attn.out_proj",
1011
+ "language_model.model.layers.12.mlp.up_proj",
1012
+ "language_model.model.layers.12.mlp.down_proj",
1013
+ "language_model.model.layers.12.mlp.gate_proj",
1014
+ "language_model.model.layers.12.linear_attn.out_proj",
1015
+ "language_model.model.layers.11.mlp.up_proj",
1016
+ "language_model.model.layers.11.mlp.down_proj",
1017
+ "language_model.model.layers.11.mlp.gate_proj",
1018
+ "language_model.model.layers.11.self_attn.o_proj",
1019
+ "language_model.model.layers.11.self_attn.v_proj",
1020
+ "language_model.model.layers.11.self_attn.k_proj",
1021
+ "language_model.model.layers.11.self_attn.q_proj",
1022
+ "language_model.model.layers.10.mlp.up_proj",
1023
+ "language_model.model.layers.10.mlp.down_proj",
1024
+ "language_model.model.layers.10.mlp.gate_proj",
1025
+ "language_model.model.layers.10.linear_attn.out_proj",
1026
+ "language_model.model.layers.9.mlp.up_proj",
1027
+ "language_model.model.layers.9.mlp.down_proj",
1028
+ "language_model.model.layers.9.mlp.gate_proj",
1029
+ "language_model.model.layers.9.linear_attn.out_proj",
1030
+ "language_model.model.layers.8.mlp.up_proj",
1031
+ "language_model.model.layers.8.mlp.down_proj",
1032
+ "language_model.model.layers.8.mlp.gate_proj",
1033
+ "language_model.model.layers.8.linear_attn.out_proj",
1034
+ "language_model.model.layers.7.mlp.up_proj",
1035
+ "language_model.model.layers.7.mlp.down_proj",
1036
+ "language_model.model.layers.7.mlp.gate_proj",
1037
+ "language_model.model.layers.7.self_attn.o_proj",
1038
+ "language_model.model.layers.7.self_attn.v_proj",
1039
+ "language_model.model.layers.7.self_attn.k_proj",
1040
+ "language_model.model.layers.7.self_attn.q_proj",
1041
+ "language_model.model.layers.6.mlp.up_proj",
1042
+ "language_model.model.layers.6.mlp.down_proj",
1043
+ "language_model.model.layers.6.mlp.gate_proj",
1044
+ "language_model.model.layers.6.linear_attn.out_proj",
1045
+ "language_model.model.layers.5.mlp.up_proj",
1046
+ "language_model.model.layers.5.mlp.down_proj",
1047
+ "language_model.model.layers.5.mlp.gate_proj",
1048
+ "language_model.model.layers.5.linear_attn.out_proj",
1049
+ "language_model.model.layers.4.mlp.up_proj",
1050
+ "language_model.model.layers.4.mlp.down_proj",
1051
+ "language_model.model.layers.4.mlp.gate_proj",
1052
+ "language_model.model.layers.4.linear_attn.out_proj",
1053
+ "language_model.model.layers.3.mlp.up_proj",
1054
+ "language_model.model.layers.3.mlp.down_proj",
1055
+ "language_model.model.layers.3.mlp.gate_proj",
1056
+ "language_model.model.layers.3.self_attn.o_proj",
1057
+ "language_model.model.layers.3.self_attn.v_proj",
1058
+ "language_model.model.layers.3.self_attn.k_proj",
1059
+ "language_model.model.layers.3.self_attn.q_proj",
1060
+ "language_model.model.layers.2.mlp.up_proj",
1061
+ "language_model.model.layers.2.mlp.down_proj",
1062
+ "language_model.model.layers.2.mlp.gate_proj",
1063
+ "language_model.model.layers.2.linear_attn.out_proj",
1064
+ "language_model.model.layers.1.mlp.up_proj",
1065
+ "language_model.model.layers.1.mlp.down_proj",
1066
+ "language_model.model.layers.1.mlp.gate_proj",
1067
+ "language_model.model.layers.1.linear_attn.out_proj",
1068
+ "language_model.model.layers.0.mlp.up_proj",
1069
+ "language_model.model.layers.0.mlp.down_proj",
1070
+ "language_model.model.layers.0.mlp.gate_proj",
1071
+ "language_model.model.layers.0.linear_attn.out_proj"
1072
+ ]
1073
+ },
1074
+ "rank": 16,
1075
+ "scale": 4.0,
1076
+ "dropout": 0.0,
1077
+ "base_model_commit_hash": "fffbc0d8711cf87df6d1f97938c5e50ef94aadc7",
1078
+ "base_resolved_quantization_map": {
1079
+ "language_model.model.layers.0.linear_attn.in_proj_qkv": {
1080
+ "bits": 4,
1081
+ "group_size": 64,
1082
+ "mode": "affine"
1083
+ },
1084
+ "language_model.model.layers.0.linear_attn.in_proj_z": {
1085
+ "bits": 4,
1086
+ "group_size": 64,
1087
+ "mode": "affine"
1088
+ },
1089
+ "language_model.model.layers.0.linear_attn.in_proj_b": {
1090
+ "bits": 4,
1091
+ "group_size": 64,
1092
+ "mode": "affine"
1093
+ },
1094
+ "language_model.model.layers.0.linear_attn.in_proj_a": {
1095
+ "bits": 4,
1096
+ "group_size": 64,
1097
+ "mode": "affine"
1098
+ },
1099
+ "language_model.model.layers.0.linear_attn.out_proj": {
1100
+ "bits": 4,
1101
+ "group_size": 64,
1102
+ "mode": "affine"
1103
+ },
1104
+ "language_model.model.layers.0.mlp.gate_proj": {
1105
+ "bits": 4,
1106
+ "group_size": 64,
1107
+ "mode": "affine"
1108
+ },
1109
+ "language_model.model.layers.0.mlp.down_proj": {
1110
+ "bits": 4,
1111
+ "group_size": 64,
1112
+ "mode": "affine"
1113
+ },
1114
+ "language_model.model.layers.0.mlp.up_proj": {
1115
+ "bits": 4,
1116
+ "group_size": 64,
1117
+ "mode": "affine"
1118
+ },
1119
+ "language_model.model.layers.1.linear_attn.in_proj_qkv": {
1120
+ "bits": 4,
1121
+ "group_size": 64,
1122
+ "mode": "affine"
1123
+ },
1124
+ "language_model.model.layers.1.linear_attn.in_proj_z": {
1125
+ "bits": 4,
1126
+ "group_size": 64,
1127
+ "mode": "affine"
1128
+ },
1129
+ "language_model.model.layers.1.linear_attn.in_proj_b": {
1130
+ "bits": 4,
1131
+ "group_size": 64,
1132
+ "mode": "affine"
1133
+ },
1134
+ "language_model.model.layers.1.linear_attn.in_proj_a": {
1135
+ "bits": 4,
1136
+ "group_size": 64,
1137
+ "mode": "affine"
1138
+ },
1139
+ "language_model.model.layers.1.linear_attn.out_proj": {
1140
+ "bits": 4,
1141
+ "group_size": 64,
1142
+ "mode": "affine"
1143
+ },
1144
+ "language_model.model.layers.1.mlp.gate_proj": {
1145
+ "bits": 4,
1146
+ "group_size": 64,
1147
+ "mode": "affine"
1148
+ },
1149
+ "language_model.model.layers.1.mlp.down_proj": {
1150
+ "bits": 4,
1151
+ "group_size": 64,
1152
+ "mode": "affine"
1153
+ },
1154
+ "language_model.model.layers.1.mlp.up_proj": {
1155
+ "bits": 4,
1156
+ "group_size": 64,
1157
+ "mode": "affine"
1158
+ },
1159
+ "language_model.model.layers.2.linear_attn.in_proj_qkv": {
1160
+ "bits": 4,
1161
+ "group_size": 64,
1162
+ "mode": "affine"
1163
+ },
1164
+ "language_model.model.layers.2.linear_attn.in_proj_z": {
1165
+ "bits": 4,
1166
+ "group_size": 64,
1167
+ "mode": "affine"
1168
+ },
1169
+ "language_model.model.layers.2.linear_attn.in_proj_b": {
1170
+ "bits": 4,
1171
+ "group_size": 64,
1172
+ "mode": "affine"
1173
+ },
1174
+ "language_model.model.layers.2.linear_attn.in_proj_a": {
1175
+ "bits": 4,
1176
+ "group_size": 64,
1177
+ "mode": "affine"
1178
+ },
1179
+ "language_model.model.layers.2.linear_attn.out_proj": {
1180
+ "bits": 4,
1181
+ "group_size": 64,
1182
+ "mode": "affine"
1183
+ },
1184
+ "language_model.model.layers.2.mlp.gate_proj": {
1185
+ "bits": 4,
1186
+ "group_size": 64,
1187
+ "mode": "affine"
1188
+ },
1189
+ "language_model.model.layers.2.mlp.down_proj": {
1190
+ "bits": 4,
1191
+ "group_size": 64,
1192
+ "mode": "affine"
1193
+ },
1194
+ "language_model.model.layers.2.mlp.up_proj": {
1195
+ "bits": 4,
1196
+ "group_size": 64,
1197
+ "mode": "affine"
1198
+ },
1199
+ "language_model.model.layers.3.self_attn.q_proj": {
1200
+ "bits": 4,
1201
+ "group_size": 64,
1202
+ "mode": "affine"
1203
+ },
1204
+ "language_model.model.layers.3.self_attn.k_proj": {
1205
+ "bits": 4,
1206
+ "group_size": 64,
1207
+ "mode": "affine"
1208
+ },
1209
+ "language_model.model.layers.3.self_attn.v_proj": {
1210
+ "bits": 4,
1211
+ "group_size": 64,
1212
+ "mode": "affine"
1213
+ },
1214
+ "language_model.model.layers.3.self_attn.o_proj": {
1215
+ "bits": 4,
1216
+ "group_size": 64,
1217
+ "mode": "affine"
1218
+ },
1219
+ "language_model.model.layers.3.mlp.gate_proj": {
1220
+ "bits": 4,
1221
+ "group_size": 64,
1222
+ "mode": "affine"
1223
+ },
1224
+ "language_model.model.layers.3.mlp.down_proj": {
1225
+ "bits": 4,
1226
+ "group_size": 64,
1227
+ "mode": "affine"
1228
+ },
1229
+ "language_model.model.layers.3.mlp.up_proj": {
1230
+ "bits": 4,
1231
+ "group_size": 64,
1232
+ "mode": "affine"
1233
+ },
1234
+ "language_model.model.layers.4.linear_attn.in_proj_qkv": {
1235
+ "bits": 4,
1236
+ "group_size": 64,
1237
+ "mode": "affine"
1238
+ },
1239
+ "language_model.model.layers.4.linear_attn.in_proj_z": {
1240
+ "bits": 4,
1241
+ "group_size": 64,
1242
+ "mode": "affine"
1243
+ },
1244
+ "language_model.model.layers.4.linear_attn.in_proj_b": {
1245
+ "bits": 4,
1246
+ "group_size": 64,
1247
+ "mode": "affine"
1248
+ },
1249
+ "language_model.model.layers.4.linear_attn.in_proj_a": {
1250
+ "bits": 4,
1251
+ "group_size": 64,
1252
+ "mode": "affine"
1253
+ },
1254
+ "language_model.model.layers.4.linear_attn.out_proj": {
1255
+ "bits": 4,
1256
+ "group_size": 64,
1257
+ "mode": "affine"
1258
+ },
1259
+ "language_model.model.layers.4.mlp.gate_proj": {
1260
+ "bits": 4,
1261
+ "group_size": 64,
1262
+ "mode": "affine"
1263
+ },
1264
+ "language_model.model.layers.4.mlp.down_proj": {
1265
+ "bits": 4,
1266
+ "group_size": 64,
1267
+ "mode": "affine"
1268
+ },
1269
+ "language_model.model.layers.4.mlp.up_proj": {
1270
+ "bits": 4,
1271
+ "group_size": 64,
1272
+ "mode": "affine"
1273
+ },
1274
+ "language_model.model.layers.5.linear_attn.in_proj_qkv": {
1275
+ "bits": 4,
1276
+ "group_size": 64,
1277
+ "mode": "affine"
1278
+ },
1279
+ "language_model.model.layers.5.linear_attn.in_proj_z": {
1280
+ "bits": 4,
1281
+ "group_size": 64,
1282
+ "mode": "affine"
1283
+ },
1284
+ "language_model.model.layers.5.linear_attn.in_proj_b": {
1285
+ "bits": 4,
1286
+ "group_size": 64,
1287
+ "mode": "affine"
1288
+ },
1289
+ "language_model.model.layers.5.linear_attn.in_proj_a": {
1290
+ "bits": 4,
1291
+ "group_size": 64,
1292
+ "mode": "affine"
1293
+ },
1294
+ "language_model.model.layers.5.linear_attn.out_proj": {
1295
+ "bits": 4,
1296
+ "group_size": 64,
1297
+ "mode": "affine"
1298
+ },
1299
+ "language_model.model.layers.5.mlp.gate_proj": {
1300
+ "bits": 4,
1301
+ "group_size": 64,
1302
+ "mode": "affine"
1303
+ },
1304
+ "language_model.model.layers.5.mlp.down_proj": {
1305
+ "bits": 4,
1306
+ "group_size": 64,
1307
+ "mode": "affine"
1308
+ },
1309
+ "language_model.model.layers.5.mlp.up_proj": {
1310
+ "bits": 4,
1311
+ "group_size": 64,
1312
+ "mode": "affine"
1313
+ },
1314
+ "language_model.model.layers.6.linear_attn.in_proj_qkv": {
1315
+ "bits": 4,
1316
+ "group_size": 64,
1317
+ "mode": "affine"
1318
+ },
1319
+ "language_model.model.layers.6.linear_attn.in_proj_z": {
1320
+ "bits": 4,
1321
+ "group_size": 64,
1322
+ "mode": "affine"
1323
+ },
1324
+ "language_model.model.layers.6.linear_attn.in_proj_b": {
1325
+ "bits": 4,
1326
+ "group_size": 64,
1327
+ "mode": "affine"
1328
+ },
1329
+ "language_model.model.layers.6.linear_attn.in_proj_a": {
1330
+ "bits": 4,
1331
+ "group_size": 64,
1332
+ "mode": "affine"
1333
+ },
1334
+ "language_model.model.layers.6.linear_attn.out_proj": {
1335
+ "bits": 4,
1336
+ "group_size": 64,
1337
+ "mode": "affine"
1338
+ },
1339
+ "language_model.model.layers.6.mlp.gate_proj": {
1340
+ "bits": 4,
1341
+ "group_size": 64,
1342
+ "mode": "affine"
1343
+ },
1344
+ "language_model.model.layers.6.mlp.down_proj": {
1345
+ "bits": 4,
1346
+ "group_size": 64,
1347
+ "mode": "affine"
1348
+ },
1349
+ "language_model.model.layers.6.mlp.up_proj": {
1350
+ "bits": 4,
1351
+ "group_size": 64,
1352
+ "mode": "affine"
1353
+ },
1354
+ "language_model.model.layers.7.self_attn.q_proj": {
1355
+ "bits": 4,
1356
+ "group_size": 64,
1357
+ "mode": "affine"
1358
+ },
1359
+ "language_model.model.layers.7.self_attn.k_proj": {
1360
+ "bits": 4,
1361
+ "group_size": 64,
1362
+ "mode": "affine"
1363
+ },
1364
+ "language_model.model.layers.7.self_attn.v_proj": {
1365
+ "bits": 4,
1366
+ "group_size": 64,
1367
+ "mode": "affine"
1368
+ },
1369
+ "language_model.model.layers.7.self_attn.o_proj": {
1370
+ "bits": 4,
1371
+ "group_size": 64,
1372
+ "mode": "affine"
1373
+ },
1374
+ "language_model.model.layers.7.mlp.gate_proj": {
1375
+ "bits": 4,
1376
+ "group_size": 64,
1377
+ "mode": "affine"
1378
+ },
1379
+ "language_model.model.layers.7.mlp.down_proj": {
1380
+ "bits": 4,
1381
+ "group_size": 64,
1382
+ "mode": "affine"
1383
+ },
1384
+ "language_model.model.layers.7.mlp.up_proj": {
1385
+ "bits": 4,
1386
+ "group_size": 64,
1387
+ "mode": "affine"
1388
+ },
1389
+ "language_model.model.layers.8.linear_attn.in_proj_qkv": {
1390
+ "bits": 4,
1391
+ "group_size": 64,
1392
+ "mode": "affine"
1393
+ },
1394
+ "language_model.model.layers.8.linear_attn.in_proj_z": {
1395
+ "bits": 4,
1396
+ "group_size": 64,
1397
+ "mode": "affine"
1398
+ },
1399
+ "language_model.model.layers.8.linear_attn.in_proj_b": {
1400
+ "bits": 4,
1401
+ "group_size": 64,
1402
+ "mode": "affine"
1403
+ },
1404
+ "language_model.model.layers.8.linear_attn.in_proj_a": {
1405
+ "bits": 4,
1406
+ "group_size": 64,
1407
+ "mode": "affine"
1408
+ },
1409
+ "language_model.model.layers.8.linear_attn.out_proj": {
1410
+ "bits": 4,
1411
+ "group_size": 64,
1412
+ "mode": "affine"
1413
+ },
1414
+ "language_model.model.layers.8.mlp.gate_proj": {
1415
+ "bits": 4,
1416
+ "group_size": 64,
1417
+ "mode": "affine"
1418
+ },
1419
+ "language_model.model.layers.8.mlp.down_proj": {
1420
+ "bits": 4,
1421
+ "group_size": 64,
1422
+ "mode": "affine"
1423
+ },
1424
+ "language_model.model.layers.8.mlp.up_proj": {
1425
+ "bits": 4,
1426
+ "group_size": 64,
1427
+ "mode": "affine"
1428
+ },
1429
+ "language_model.model.layers.9.linear_attn.in_proj_qkv": {
1430
+ "bits": 4,
1431
+ "group_size": 64,
1432
+ "mode": "affine"
1433
+ },
1434
+ "language_model.model.layers.9.linear_attn.in_proj_z": {
1435
+ "bits": 4,
1436
+ "group_size": 64,
1437
+ "mode": "affine"
1438
+ },
1439
+ "language_model.model.layers.9.linear_attn.in_proj_b": {
1440
+ "bits": 4,
1441
+ "group_size": 64,
1442
+ "mode": "affine"
1443
+ },
1444
+ "language_model.model.layers.9.linear_attn.in_proj_a": {
1445
+ "bits": 4,
1446
+ "group_size": 64,
1447
+ "mode": "affine"
1448
+ },
1449
+ "language_model.model.layers.9.linear_attn.out_proj": {
1450
+ "bits": 4,
1451
+ "group_size": 64,
1452
+ "mode": "affine"
1453
+ },
1454
+ "language_model.model.layers.9.mlp.gate_proj": {
1455
+ "bits": 4,
1456
+ "group_size": 64,
1457
+ "mode": "affine"
1458
+ },
1459
+ "language_model.model.layers.9.mlp.down_proj": {
1460
+ "bits": 4,
1461
+ "group_size": 64,
1462
+ "mode": "affine"
1463
+ },
1464
+ "language_model.model.layers.9.mlp.up_proj": {
1465
+ "bits": 4,
1466
+ "group_size": 64,
1467
+ "mode": "affine"
1468
+ },
1469
+ "language_model.model.layers.10.linear_attn.in_proj_qkv": {
1470
+ "bits": 4,
1471
+ "group_size": 64,
1472
+ "mode": "affine"
1473
+ },
1474
+ "language_model.model.layers.10.linear_attn.in_proj_z": {
1475
+ "bits": 4,
1476
+ "group_size": 64,
1477
+ "mode": "affine"
1478
+ },
1479
+ "language_model.model.layers.10.linear_attn.in_proj_b": {
1480
+ "bits": 4,
1481
+ "group_size": 64,
1482
+ "mode": "affine"
1483
+ },
1484
+ "language_model.model.layers.10.linear_attn.in_proj_a": {
1485
+ "bits": 4,
1486
+ "group_size": 64,
1487
+ "mode": "affine"
1488
+ },
1489
+ "language_model.model.layers.10.linear_attn.out_proj": {
1490
+ "bits": 4,
1491
+ "group_size": 64,
1492
+ "mode": "affine"
1493
+ },
1494
+ "language_model.model.layers.10.mlp.gate_proj": {
1495
+ "bits": 4,
1496
+ "group_size": 64,
1497
+ "mode": "affine"
1498
+ },
1499
+ "language_model.model.layers.10.mlp.down_proj": {
1500
+ "bits": 4,
1501
+ "group_size": 64,
1502
+ "mode": "affine"
1503
+ },
1504
+ "language_model.model.layers.10.mlp.up_proj": {
1505
+ "bits": 4,
1506
+ "group_size": 64,
1507
+ "mode": "affine"
1508
+ },
1509
+ "language_model.model.layers.11.self_attn.q_proj": {
1510
+ "bits": 4,
1511
+ "group_size": 64,
1512
+ "mode": "affine"
1513
+ },
1514
+ "language_model.model.layers.11.self_attn.k_proj": {
1515
+ "bits": 4,
1516
+ "group_size": 64,
1517
+ "mode": "affine"
1518
+ },
1519
+ "language_model.model.layers.11.self_attn.v_proj": {
1520
+ "bits": 4,
1521
+ "group_size": 64,
1522
+ "mode": "affine"
1523
+ },
1524
+ "language_model.model.layers.11.self_attn.o_proj": {
1525
+ "bits": 4,
1526
+ "group_size": 64,
1527
+ "mode": "affine"
1528
+ },
1529
+ "language_model.model.layers.11.mlp.gate_proj": {
1530
+ "bits": 4,
1531
+ "group_size": 64,
1532
+ "mode": "affine"
1533
+ },
1534
+ "language_model.model.layers.11.mlp.down_proj": {
1535
+ "bits": 4,
1536
+ "group_size": 64,
1537
+ "mode": "affine"
1538
+ },
1539
+ "language_model.model.layers.11.mlp.up_proj": {
1540
+ "bits": 4,
1541
+ "group_size": 64,
1542
+ "mode": "affine"
1543
+ },
1544
+ "language_model.model.layers.12.linear_attn.in_proj_qkv": {
1545
+ "bits": 4,
1546
+ "group_size": 64,
1547
+ "mode": "affine"
1548
+ },
1549
+ "language_model.model.layers.12.linear_attn.in_proj_z": {
1550
+ "bits": 4,
1551
+ "group_size": 64,
1552
+ "mode": "affine"
1553
+ },
1554
+ "language_model.model.layers.12.linear_attn.in_proj_b": {
1555
+ "bits": 4,
1556
+ "group_size": 64,
1557
+ "mode": "affine"
1558
+ },
1559
+ "language_model.model.layers.12.linear_attn.in_proj_a": {
1560
+ "bits": 4,
1561
+ "group_size": 64,
1562
+ "mode": "affine"
1563
+ },
1564
+ "language_model.model.layers.12.linear_attn.out_proj": {
1565
+ "bits": 4,
1566
+ "group_size": 64,
1567
+ "mode": "affine"
1568
+ },
1569
+ "language_model.model.layers.12.mlp.gate_proj": {
1570
+ "bits": 4,
1571
+ "group_size": 64,
1572
+ "mode": "affine"
1573
+ },
1574
+ "language_model.model.layers.12.mlp.down_proj": {
1575
+ "bits": 4,
1576
+ "group_size": 64,
1577
+ "mode": "affine"
1578
+ },
1579
+ "language_model.model.layers.12.mlp.up_proj": {
1580
+ "bits": 4,
1581
+ "group_size": 64,
1582
+ "mode": "affine"
1583
+ },
1584
+ "language_model.model.layers.13.linear_attn.in_proj_qkv": {
1585
+ "bits": 4,
1586
+ "group_size": 64,
1587
+ "mode": "affine"
1588
+ },
1589
+ "language_model.model.layers.13.linear_attn.in_proj_z": {
1590
+ "bits": 4,
1591
+ "group_size": 64,
1592
+ "mode": "affine"
1593
+ },
1594
+ "language_model.model.layers.13.linear_attn.in_proj_b": {
1595
+ "bits": 4,
1596
+ "group_size": 64,
1597
+ "mode": "affine"
1598
+ },
1599
+ "language_model.model.layers.13.linear_attn.in_proj_a": {
1600
+ "bits": 4,
1601
+ "group_size": 64,
1602
+ "mode": "affine"
1603
+ },
1604
+ "language_model.model.layers.13.linear_attn.out_proj": {
1605
+ "bits": 4,
1606
+ "group_size": 64,
1607
+ "mode": "affine"
1608
+ },
1609
+ "language_model.model.layers.13.mlp.gate_proj": {
1610
+ "bits": 4,
1611
+ "group_size": 64,
1612
+ "mode": "affine"
1613
+ },
1614
+ "language_model.model.layers.13.mlp.down_proj": {
1615
+ "bits": 4,
1616
+ "group_size": 64,
1617
+ "mode": "affine"
1618
+ },
1619
+ "language_model.model.layers.13.mlp.up_proj": {
1620
+ "bits": 4,
1621
+ "group_size": 64,
1622
+ "mode": "affine"
1623
+ },
1624
+ "language_model.model.layers.14.linear_attn.in_proj_qkv": {
1625
+ "bits": 4,
1626
+ "group_size": 64,
1627
+ "mode": "affine"
1628
+ },
1629
+ "language_model.model.layers.14.linear_attn.in_proj_z": {
1630
+ "bits": 4,
1631
+ "group_size": 64,
1632
+ "mode": "affine"
1633
+ },
1634
+ "language_model.model.layers.14.linear_attn.in_proj_b": {
1635
+ "bits": 4,
1636
+ "group_size": 64,
1637
+ "mode": "affine"
1638
+ },
1639
+ "language_model.model.layers.14.linear_attn.in_proj_a": {
1640
+ "bits": 4,
1641
+ "group_size": 64,
1642
+ "mode": "affine"
1643
+ },
1644
+ "language_model.model.layers.14.linear_attn.out_proj": {
1645
+ "bits": 4,
1646
+ "group_size": 64,
1647
+ "mode": "affine"
1648
+ },
1649
+ "language_model.model.layers.14.mlp.gate_proj": {
1650
+ "bits": 4,
1651
+ "group_size": 64,
1652
+ "mode": "affine"
1653
+ },
1654
+ "language_model.model.layers.14.mlp.down_proj": {
1655
+ "bits": 4,
1656
+ "group_size": 64,
1657
+ "mode": "affine"
1658
+ },
1659
+ "language_model.model.layers.14.mlp.up_proj": {
1660
+ "bits": 4,
1661
+ "group_size": 64,
1662
+ "mode": "affine"
1663
+ },
1664
+ "language_model.model.layers.15.self_attn.q_proj": {
1665
+ "bits": 4,
1666
+ "group_size": 64,
1667
+ "mode": "affine"
1668
+ },
1669
+ "language_model.model.layers.15.self_attn.k_proj": {
1670
+ "bits": 4,
1671
+ "group_size": 64,
1672
+ "mode": "affine"
1673
+ },
1674
+ "language_model.model.layers.15.self_attn.v_proj": {
1675
+ "bits": 4,
1676
+ "group_size": 64,
1677
+ "mode": "affine"
1678
+ },
1679
+ "language_model.model.layers.15.self_attn.o_proj": {
1680
+ "bits": 4,
1681
+ "group_size": 64,
1682
+ "mode": "affine"
1683
+ },
1684
+ "language_model.model.layers.15.mlp.gate_proj": {
1685
+ "bits": 4,
1686
+ "group_size": 64,
1687
+ "mode": "affine"
1688
+ },
1689
+ "language_model.model.layers.15.mlp.down_proj": {
1690
+ "bits": 4,
1691
+ "group_size": 64,
1692
+ "mode": "affine"
1693
+ },
1694
+ "language_model.model.layers.15.mlp.up_proj": {
1695
+ "bits": 4,
1696
+ "group_size": 64,
1697
+ "mode": "affine"
1698
+ },
1699
+ "language_model.model.layers.16.linear_attn.in_proj_qkv": {
1700
+ "bits": 4,
1701
+ "group_size": 64,
1702
+ "mode": "affine"
1703
+ },
1704
+ "language_model.model.layers.16.linear_attn.in_proj_z": {
1705
+ "bits": 4,
1706
+ "group_size": 64,
1707
+ "mode": "affine"
1708
+ },
1709
+ "language_model.model.layers.16.linear_attn.in_proj_b": {
1710
+ "bits": 4,
1711
+ "group_size": 64,
1712
+ "mode": "affine"
1713
+ },
1714
+ "language_model.model.layers.16.linear_attn.in_proj_a": {
1715
+ "bits": 4,
1716
+ "group_size": 64,
1717
+ "mode": "affine"
1718
+ },
1719
+ "language_model.model.layers.16.linear_attn.out_proj": {
1720
+ "bits": 4,
1721
+ "group_size": 64,
1722
+ "mode": "affine"
1723
+ },
1724
+ "language_model.model.layers.16.mlp.gate_proj": {
1725
+ "bits": 4,
1726
+ "group_size": 64,
1727
+ "mode": "affine"
1728
+ },
1729
+ "language_model.model.layers.16.mlp.down_proj": {
1730
+ "bits": 4,
1731
+ "group_size": 64,
1732
+ "mode": "affine"
1733
+ },
1734
+ "language_model.model.layers.16.mlp.up_proj": {
1735
+ "bits": 4,
1736
+ "group_size": 64,
1737
+ "mode": "affine"
1738
+ },
1739
+ "language_model.model.layers.17.linear_attn.in_proj_qkv": {
1740
+ "bits": 4,
1741
+ "group_size": 64,
1742
+ "mode": "affine"
1743
+ },
1744
+ "language_model.model.layers.17.linear_attn.in_proj_z": {
1745
+ "bits": 4,
1746
+ "group_size": 64,
1747
+ "mode": "affine"
1748
+ },
1749
+ "language_model.model.layers.17.linear_attn.in_proj_b": {
1750
+ "bits": 4,
1751
+ "group_size": 64,
1752
+ "mode": "affine"
1753
+ },
1754
+ "language_model.model.layers.17.linear_attn.in_proj_a": {
1755
+ "bits": 4,
1756
+ "group_size": 64,
1757
+ "mode": "affine"
1758
+ },
1759
+ "language_model.model.layers.17.linear_attn.out_proj": {
1760
+ "bits": 4,
1761
+ "group_size": 64,
1762
+ "mode": "affine"
1763
+ },
1764
+ "language_model.model.layers.17.mlp.gate_proj": {
1765
+ "bits": 4,
1766
+ "group_size": 64,
1767
+ "mode": "affine"
1768
+ },
1769
+ "language_model.model.layers.17.mlp.down_proj": {
1770
+ "bits": 4,
1771
+ "group_size": 64,
1772
+ "mode": "affine"
1773
+ },
1774
+ "language_model.model.layers.17.mlp.up_proj": {
1775
+ "bits": 4,
1776
+ "group_size": 64,
1777
+ "mode": "affine"
1778
+ },
1779
+ "language_model.model.layers.18.linear_attn.in_proj_qkv": {
1780
+ "bits": 4,
1781
+ "group_size": 64,
1782
+ "mode": "affine"
1783
+ },
1784
+ "language_model.model.layers.18.linear_attn.in_proj_z": {
1785
+ "bits": 4,
1786
+ "group_size": 64,
1787
+ "mode": "affine"
1788
+ },
1789
+ "language_model.model.layers.18.linear_attn.in_proj_b": {
1790
+ "bits": 4,
1791
+ "group_size": 64,
1792
+ "mode": "affine"
1793
+ },
1794
+ "language_model.model.layers.18.linear_attn.in_proj_a": {
1795
+ "bits": 4,
1796
+ "group_size": 64,
1797
+ "mode": "affine"
1798
+ },
1799
+ "language_model.model.layers.18.linear_attn.out_proj": {
1800
+ "bits": 4,
1801
+ "group_size": 64,
1802
+ "mode": "affine"
1803
+ },
1804
+ "language_model.model.layers.18.mlp.gate_proj": {
1805
+ "bits": 4,
1806
+ "group_size": 64,
1807
+ "mode": "affine"
1808
+ },
1809
+ "language_model.model.layers.18.mlp.down_proj": {
1810
+ "bits": 4,
1811
+ "group_size": 64,
1812
+ "mode": "affine"
1813
+ },
1814
+ "language_model.model.layers.18.mlp.up_proj": {
1815
+ "bits": 4,
1816
+ "group_size": 64,
1817
+ "mode": "affine"
1818
+ },
1819
+ "language_model.model.layers.19.self_attn.q_proj": {
1820
+ "bits": 4,
1821
+ "group_size": 64,
1822
+ "mode": "affine"
1823
+ },
1824
+ "language_model.model.layers.19.self_attn.k_proj": {
1825
+ "bits": 4,
1826
+ "group_size": 64,
1827
+ "mode": "affine"
1828
+ },
1829
+ "language_model.model.layers.19.self_attn.v_proj": {
1830
+ "bits": 4,
1831
+ "group_size": 64,
1832
+ "mode": "affine"
1833
+ },
1834
+ "language_model.model.layers.19.self_attn.o_proj": {
1835
+ "bits": 4,
1836
+ "group_size": 64,
1837
+ "mode": "affine"
1838
+ },
1839
+ "language_model.model.layers.19.mlp.gate_proj": {
1840
+ "bits": 4,
1841
+ "group_size": 64,
1842
+ "mode": "affine"
1843
+ },
1844
+ "language_model.model.layers.19.mlp.down_proj": {
1845
+ "bits": 4,
1846
+ "group_size": 64,
1847
+ "mode": "affine"
1848
+ },
1849
+ "language_model.model.layers.19.mlp.up_proj": {
1850
+ "bits": 4,
1851
+ "group_size": 64,
1852
+ "mode": "affine"
1853
+ },
1854
+ "language_model.model.layers.20.linear_attn.in_proj_qkv": {
1855
+ "bits": 4,
1856
+ "group_size": 64,
1857
+ "mode": "affine"
1858
+ },
1859
+ "language_model.model.layers.20.linear_attn.in_proj_z": {
1860
+ "bits": 4,
1861
+ "group_size": 64,
1862
+ "mode": "affine"
1863
+ },
1864
+ "language_model.model.layers.20.linear_attn.in_proj_b": {
1865
+ "bits": 4,
1866
+ "group_size": 64,
1867
+ "mode": "affine"
1868
+ },
1869
+ "language_model.model.layers.20.linear_attn.in_proj_a": {
1870
+ "bits": 4,
1871
+ "group_size": 64,
1872
+ "mode": "affine"
1873
+ },
1874
+ "language_model.model.layers.20.linear_attn.out_proj": {
1875
+ "bits": 4,
1876
+ "group_size": 64,
1877
+ "mode": "affine"
1878
+ },
1879
+ "language_model.model.layers.20.mlp.gate_proj": {
1880
+ "bits": 4,
1881
+ "group_size": 64,
1882
+ "mode": "affine"
1883
+ },
1884
+ "language_model.model.layers.20.mlp.down_proj": {
1885
+ "bits": 4,
1886
+ "group_size": 64,
1887
+ "mode": "affine"
1888
+ },
1889
+ "language_model.model.layers.20.mlp.up_proj": {
1890
+ "bits": 4,
1891
+ "group_size": 64,
1892
+ "mode": "affine"
1893
+ },
1894
+ "language_model.model.layers.21.linear_attn.in_proj_qkv": {
1895
+ "bits": 4,
1896
+ "group_size": 64,
1897
+ "mode": "affine"
1898
+ },
1899
+ "language_model.model.layers.21.linear_attn.in_proj_z": {
1900
+ "bits": 4,
1901
+ "group_size": 64,
1902
+ "mode": "affine"
1903
+ },
1904
+ "language_model.model.layers.21.linear_attn.in_proj_b": {
1905
+ "bits": 4,
1906
+ "group_size": 64,
1907
+ "mode": "affine"
1908
+ },
1909
+ "language_model.model.layers.21.linear_attn.in_proj_a": {
1910
+ "bits": 4,
1911
+ "group_size": 64,
1912
+ "mode": "affine"
1913
+ },
1914
+ "language_model.model.layers.21.linear_attn.out_proj": {
1915
+ "bits": 4,
1916
+ "group_size": 64,
1917
+ "mode": "affine"
1918
+ },
1919
+ "language_model.model.layers.21.mlp.gate_proj": {
1920
+ "bits": 4,
1921
+ "group_size": 64,
1922
+ "mode": "affine"
1923
+ },
1924
+ "language_model.model.layers.21.mlp.down_proj": {
1925
+ "bits": 4,
1926
+ "group_size": 64,
1927
+ "mode": "affine"
1928
+ },
1929
+ "language_model.model.layers.21.mlp.up_proj": {
1930
+ "bits": 4,
1931
+ "group_size": 64,
1932
+ "mode": "affine"
1933
+ },
1934
+ "language_model.model.layers.22.linear_attn.in_proj_qkv": {
1935
+ "bits": 4,
1936
+ "group_size": 64,
1937
+ "mode": "affine"
1938
+ },
1939
+ "language_model.model.layers.22.linear_attn.in_proj_z": {
1940
+ "bits": 4,
1941
+ "group_size": 64,
1942
+ "mode": "affine"
1943
+ },
1944
+ "language_model.model.layers.22.linear_attn.in_proj_b": {
1945
+ "bits": 4,
1946
+ "group_size": 64,
1947
+ "mode": "affine"
1948
+ },
1949
+ "language_model.model.layers.22.linear_attn.in_proj_a": {
1950
+ "bits": 4,
1951
+ "group_size": 64,
1952
+ "mode": "affine"
1953
+ },
1954
+ "language_model.model.layers.22.linear_attn.out_proj": {
1955
+ "bits": 4,
1956
+ "group_size": 64,
1957
+ "mode": "affine"
1958
+ },
1959
+ "language_model.model.layers.22.mlp.gate_proj": {
1960
+ "bits": 4,
1961
+ "group_size": 64,
1962
+ "mode": "affine"
1963
+ },
1964
+ "language_model.model.layers.22.mlp.down_proj": {
1965
+ "bits": 4,
1966
+ "group_size": 64,
1967
+ "mode": "affine"
1968
+ },
1969
+ "language_model.model.layers.22.mlp.up_proj": {
1970
+ "bits": 4,
1971
+ "group_size": 64,
1972
+ "mode": "affine"
1973
+ },
1974
+ "language_model.model.layers.23.self_attn.q_proj": {
1975
+ "bits": 4,
1976
+ "group_size": 64,
1977
+ "mode": "affine"
1978
+ },
1979
+ "language_model.model.layers.23.self_attn.k_proj": {
1980
+ "bits": 4,
1981
+ "group_size": 64,
1982
+ "mode": "affine"
1983
+ },
1984
+ "language_model.model.layers.23.self_attn.v_proj": {
1985
+ "bits": 4,
1986
+ "group_size": 64,
1987
+ "mode": "affine"
1988
+ },
1989
+ "language_model.model.layers.23.self_attn.o_proj": {
1990
+ "bits": 4,
1991
+ "group_size": 64,
1992
+ "mode": "affine"
1993
+ },
1994
+ "language_model.model.layers.23.mlp.gate_proj": {
1995
+ "bits": 4,
1996
+ "group_size": 64,
1997
+ "mode": "affine"
1998
+ },
1999
+ "language_model.model.layers.23.mlp.down_proj": {
2000
+ "bits": 4,
2001
+ "group_size": 64,
2002
+ "mode": "affine"
2003
+ },
2004
+ "language_model.model.layers.23.mlp.up_proj": {
2005
+ "bits": 4,
2006
+ "group_size": 64,
2007
+ "mode": "affine"
2008
+ }
2009
+ },
2010
+ "requires_unsloth_mlx_runtime_quantization": true,
2011
+ "unsloth_mlx_lora_module_paths": [
2012
+ "language_model.model.layers.23.mlp.up_proj",
2013
+ "language_model.model.layers.23.mlp.down_proj",
2014
+ "language_model.model.layers.23.mlp.gate_proj",
2015
+ "language_model.model.layers.23.self_attn.o_proj",
2016
+ "language_model.model.layers.23.self_attn.v_proj",
2017
+ "language_model.model.layers.23.self_attn.k_proj",
2018
+ "language_model.model.layers.23.self_attn.q_proj",
2019
+ "language_model.model.layers.22.mlp.up_proj",
2020
+ "language_model.model.layers.22.mlp.down_proj",
2021
+ "language_model.model.layers.22.mlp.gate_proj",
2022
+ "language_model.model.layers.22.linear_attn.out_proj",
2023
+ "language_model.model.layers.21.mlp.up_proj",
2024
+ "language_model.model.layers.21.mlp.down_proj",
2025
+ "language_model.model.layers.21.mlp.gate_proj",
2026
+ "language_model.model.layers.21.linear_attn.out_proj",
2027
+ "language_model.model.layers.20.mlp.up_proj",
2028
+ "language_model.model.layers.20.mlp.down_proj",
2029
+ "language_model.model.layers.20.mlp.gate_proj",
2030
+ "language_model.model.layers.20.linear_attn.out_proj",
2031
+ "language_model.model.layers.19.mlp.up_proj",
2032
+ "language_model.model.layers.19.mlp.down_proj",
2033
+ "language_model.model.layers.19.mlp.gate_proj",
2034
+ "language_model.model.layers.19.self_attn.o_proj",
2035
+ "language_model.model.layers.19.self_attn.v_proj",
2036
+ "language_model.model.layers.19.self_attn.k_proj",
2037
+ "language_model.model.layers.19.self_attn.q_proj",
2038
+ "language_model.model.layers.18.mlp.up_proj",
2039
+ "language_model.model.layers.18.mlp.down_proj",
2040
+ "language_model.model.layers.18.mlp.gate_proj",
2041
+ "language_model.model.layers.18.linear_attn.out_proj",
2042
+ "language_model.model.layers.17.mlp.up_proj",
2043
+ "language_model.model.layers.17.mlp.down_proj",
2044
+ "language_model.model.layers.17.mlp.gate_proj",
2045
+ "language_model.model.layers.17.linear_attn.out_proj",
2046
+ "language_model.model.layers.16.mlp.up_proj",
2047
+ "language_model.model.layers.16.mlp.down_proj",
2048
+ "language_model.model.layers.16.mlp.gate_proj",
2049
+ "language_model.model.layers.16.linear_attn.out_proj",
2050
+ "language_model.model.layers.15.mlp.up_proj",
2051
+ "language_model.model.layers.15.mlp.down_proj",
2052
+ "language_model.model.layers.15.mlp.gate_proj",
2053
+ "language_model.model.layers.15.self_attn.o_proj",
2054
+ "language_model.model.layers.15.self_attn.v_proj",
2055
+ "language_model.model.layers.15.self_attn.k_proj",
2056
+ "language_model.model.layers.15.self_attn.q_proj",
2057
+ "language_model.model.layers.14.mlp.up_proj",
2058
+ "language_model.model.layers.14.mlp.down_proj",
2059
+ "language_model.model.layers.14.mlp.gate_proj",
2060
+ "language_model.model.layers.14.linear_attn.out_proj",
2061
+ "language_model.model.layers.13.mlp.up_proj",
2062
+ "language_model.model.layers.13.mlp.down_proj",
2063
+ "language_model.model.layers.13.mlp.gate_proj",
2064
+ "language_model.model.layers.13.linear_attn.out_proj",
2065
+ "language_model.model.layers.12.mlp.up_proj",
2066
+ "language_model.model.layers.12.mlp.down_proj",
2067
+ "language_model.model.layers.12.mlp.gate_proj",
2068
+ "language_model.model.layers.12.linear_attn.out_proj",
2069
+ "language_model.model.layers.11.mlp.up_proj",
2070
+ "language_model.model.layers.11.mlp.down_proj",
2071
+ "language_model.model.layers.11.mlp.gate_proj",
2072
+ "language_model.model.layers.11.self_attn.o_proj",
2073
+ "language_model.model.layers.11.self_attn.v_proj",
2074
+ "language_model.model.layers.11.self_attn.k_proj",
2075
+ "language_model.model.layers.11.self_attn.q_proj",
2076
+ "language_model.model.layers.10.mlp.up_proj",
2077
+ "language_model.model.layers.10.mlp.down_proj",
2078
+ "language_model.model.layers.10.mlp.gate_proj",
2079
+ "language_model.model.layers.10.linear_attn.out_proj",
2080
+ "language_model.model.layers.9.mlp.up_proj",
2081
+ "language_model.model.layers.9.mlp.down_proj",
2082
+ "language_model.model.layers.9.mlp.gate_proj",
2083
+ "language_model.model.layers.9.linear_attn.out_proj",
2084
+ "language_model.model.layers.8.mlp.up_proj",
2085
+ "language_model.model.layers.8.mlp.down_proj",
2086
+ "language_model.model.layers.8.mlp.gate_proj",
2087
+ "language_model.model.layers.8.linear_attn.out_proj",
2088
+ "language_model.model.layers.7.mlp.up_proj",
2089
+ "language_model.model.layers.7.mlp.down_proj",
2090
+ "language_model.model.layers.7.mlp.gate_proj",
2091
+ "language_model.model.layers.7.self_attn.o_proj",
2092
+ "language_model.model.layers.7.self_attn.v_proj",
2093
+ "language_model.model.layers.7.self_attn.k_proj",
2094
+ "language_model.model.layers.7.self_attn.q_proj",
2095
+ "language_model.model.layers.6.mlp.up_proj",
2096
+ "language_model.model.layers.6.mlp.down_proj",
2097
+ "language_model.model.layers.6.mlp.gate_proj",
2098
+ "language_model.model.layers.6.linear_attn.out_proj",
2099
+ "language_model.model.layers.5.mlp.up_proj",
2100
+ "language_model.model.layers.5.mlp.down_proj",
2101
+ "language_model.model.layers.5.mlp.gate_proj",
2102
+ "language_model.model.layers.5.linear_attn.out_proj",
2103
+ "language_model.model.layers.4.mlp.up_proj",
2104
+ "language_model.model.layers.4.mlp.down_proj",
2105
+ "language_model.model.layers.4.mlp.gate_proj",
2106
+ "language_model.model.layers.4.linear_attn.out_proj",
2107
+ "language_model.model.layers.3.mlp.up_proj",
2108
+ "language_model.model.layers.3.mlp.down_proj",
2109
+ "language_model.model.layers.3.mlp.gate_proj",
2110
+ "language_model.model.layers.3.self_attn.o_proj",
2111
+ "language_model.model.layers.3.self_attn.v_proj",
2112
+ "language_model.model.layers.3.self_attn.k_proj",
2113
+ "language_model.model.layers.3.self_attn.q_proj",
2114
+ "language_model.model.layers.2.mlp.up_proj",
2115
+ "language_model.model.layers.2.mlp.down_proj",
2116
+ "language_model.model.layers.2.mlp.gate_proj",
2117
+ "language_model.model.layers.2.linear_attn.out_proj",
2118
+ "language_model.model.layers.1.mlp.up_proj",
2119
+ "language_model.model.layers.1.mlp.down_proj",
2120
+ "language_model.model.layers.1.mlp.gate_proj",
2121
+ "language_model.model.layers.1.linear_attn.out_proj",
2122
+ "language_model.model.layers.0.mlp.up_proj",
2123
+ "language_model.model.layers.0.mlp.down_proj",
2124
+ "language_model.model.layers.0.mlp.gate_proj",
2125
+ "language_model.model.layers.0.linear_attn.out_proj"
2126
+ ]
2127
+ }
adapters.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0adc78bae5a0204220ef11bf1b4685fe320f51ae85061af3fa1b67fa13f1db92
3
+ size 48393720
chat_template.jinja ADDED
@@ -0,0 +1,158 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- set image_count = namespace(value=0) %}
2
+ {%- set video_count = namespace(value=0) %}
3
+ {%- macro render_content(content, do_vision_count, is_system_content=false) %}
4
+ {%- if content is string %}
5
+ {{- content }}
6
+ {%- elif content is iterable and content is not mapping %}
7
+ {%- for item in content %}
8
+ {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
9
+ {%- if is_system_content %}
10
+ {{- raise_exception('System message cannot contain images.') }}
11
+ {%- endif %}
12
+ {%- if do_vision_count %}
13
+ {%- set image_count.value = image_count.value + 1 %}
14
+ {%- endif %}
15
+ {%- if add_vision_id %}
16
+ {{- 'Picture ' ~ image_count.value ~ ': ' }}
17
+ {%- endif %}
18
+ {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
19
+ {%- elif 'video' in item or item.type == 'video' %}
20
+ {%- if is_system_content %}
21
+ {{- raise_exception('System message cannot contain videos.') }}
22
+ {%- endif %}
23
+ {%- if do_vision_count %}
24
+ {%- set video_count.value = video_count.value + 1 %}
25
+ {%- endif %}
26
+ {%- if add_vision_id %}
27
+ {{- 'Video ' ~ video_count.value ~ ': ' }}
28
+ {%- endif %}
29
+ {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
30
+ {%- elif 'text' in item %}
31
+ {{- item.text }}
32
+ {%- else %}
33
+ {{- raise_exception('Unexpected item type in content.') }}
34
+ {%- endif %}
35
+ {%- endfor %}
36
+ {%- elif content is none or content is undefined %}
37
+ {{- '' }}
38
+ {%- else %}
39
+ {{- raise_exception('Unexpected content type.') }}
40
+ {%- endif %}
41
+ {%- endmacro %}
42
+ {%- if not messages %}
43
+ {{- raise_exception('No messages provided.') }}
44
+ {%- endif %}
45
+ {%- set num_sys = 0 %}
46
+ {%- set merged_system = '' %}
47
+ {%- if messages[0].role == 'system' or messages[0].role == 'developer' %}
48
+ {%- set first = render_content(messages[0].content, false, true)|trim %}
49
+ {%- if messages|length > 1 and (messages[1].role == 'system' or messages[1].role == 'developer') %}
50
+ {%- set second = render_content(messages[1].content, false, true)|trim %}
51
+ {%- set merged_system = first + '\n' + second %}
52
+ {%- set num_sys = 2 %}
53
+ {%- else %}
54
+ {%- set merged_system = first %}
55
+ {%- set num_sys = 1 %}
56
+ {%- endif %}
57
+ {%- endif %}
58
+ {%- if tools and tools is iterable and tools is not mapping %}
59
+ {{- '<|im_start|>system\n' }}
60
+ {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
61
+ {%- for tool in tools %}
62
+ {{- "\n" }}
63
+ {{- tool | tojson }}
64
+ {%- endfor %}
65
+ {{- "\n</tools>" }}
66
+ {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
67
+ {%- if merged_system %}
68
+ {{- '\n\n' + merged_system }}
69
+ {%- endif %}
70
+ {{- '<|im_end|>\n' }}
71
+ {%- else %}
72
+ {%- if merged_system %}
73
+ {{- '<|im_start|>system\n' + merged_system + '<|im_end|>\n' }}
74
+ {%- endif %}
75
+ {%- endif %}
76
+ {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
77
+ {%- for message in messages[::-1] %}
78
+ {%- set index = (messages|length - 1) - loop.index0 %}
79
+ {%- if ns.multi_step_tool and message.role == "user" %}
80
+ {%- set content = render_content(message.content, false)|trim %}
81
+ {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
82
+ {%- set ns.multi_step_tool = false %}
83
+ {%- set ns.last_query_index = index %}
84
+ {%- endif %}
85
+ {%- endif %}
86
+ {%- endfor %}
87
+ {%- for message in messages %}
88
+ {%- if loop.index0 >= num_sys and message.role != "system" and message.role != "developer" %}
89
+ {%- set content = render_content(message.content, true)|trim %}
90
+ {%- if message.role == "user" %}
91
+ {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
92
+ {%- elif message.role == "assistant" %}
93
+ {%- set reasoning_content = '' %}
94
+ {%- if message.reasoning_content is string %}
95
+ {%- set reasoning_content = message.reasoning_content %}
96
+ {%- else %}
97
+ {%- if '</think>' in content %}
98
+ {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
99
+ {%- set content = content.split('</think>')[-1].lstrip('\n') %}
100
+ {%- endif %}
101
+ {%- endif %}
102
+ {%- set reasoning_content = reasoning_content|trim %}
103
+ {%- if loop.index0 > ns.last_query_index %}
104
+ {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
105
+ {%- else %}
106
+ {{- '<|im_start|>' + message.role + '\n' + content }}
107
+ {%- endif %}
108
+ {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
109
+ {%- for tool_call in message.tool_calls %}
110
+ {%- if tool_call.function is defined %}
111
+ {%- set tool_call = tool_call.function %}
112
+ {%- endif %}
113
+ {%- if loop.first %}
114
+ {%- if content|trim %}
115
+ {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
116
+ {%- else %}
117
+ {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
118
+ {%- endif %}
119
+ {%- else %}
120
+ {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
121
+ {%- endif %}
122
+ {%- if tool_call.arguments is mapping %}
123
+ {%- for args_name in tool_call.arguments %}
124
+ {%- set args_value = tool_call.arguments[args_name] %}
125
+ {{- '<parameter=' + args_name + '>\n' }}
126
+ {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
127
+ {{- args_value }}
128
+ {{- '\n</parameter>\n' }}
129
+ {%- endfor %}
130
+ {%- endif %}
131
+ {{- '</function>\n</tool_call>' }}
132
+ {%- endfor %}
133
+ {%- endif %}
134
+ {{- '<|im_end|>\n' }}
135
+ {%- elif message.role == "tool" %}
136
+ {%- if loop.previtem and loop.previtem.role != "tool" %}
137
+ {{- '<|im_start|>user' }}
138
+ {%- endif %}
139
+ {{- '\n<tool_response>\n' }}
140
+ {{- content }}
141
+ {{- '\n</tool_response>' }}
142
+ {%- if not loop.last and loop.nextitem.role != "tool" %}
143
+ {{- '<|im_end|>\n' }}
144
+ {%- elif loop.last %}
145
+ {{- '<|im_end|>\n' }}
146
+ {%- endif %}
147
+ {%- endif %}
148
+ {%- endif %}
149
+ {%- endfor %}
150
+ {%- if add_generation_prompt %}
151
+ {{- '<|im_start|>assistant\n' }}
152
+ {%- if enable_thinking is defined and enable_thinking is true %}
153
+ {{- '<think>\n' }}
154
+ {%- else %}
155
+ {{- '<think>\n\n</think>\n\n' }}
156
+ {%- endif %}
157
+ {%- endif %}
158
+ {#- Unsloth fixes - developer role, tool calling #}
config.json ADDED
@@ -0,0 +1,98 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen3_5ForConditionalGeneration"
4
+ ],
5
+ "image_token_id": 248056,
6
+ "model_type": "qwen3_5",
7
+ "text_config": {
8
+ "attention_bias": false,
9
+ "attention_dropout": 0.0,
10
+ "attn_output_gate": true,
11
+ "torch_dtype": "bfloat16",
12
+ "eos_token_id": 248044,
13
+ "full_attention_interval": 4,
14
+ "head_dim": 256,
15
+ "hidden_act": "silu",
16
+ "hidden_size": 2048,
17
+ "initializer_range": 0.02,
18
+ "intermediate_size": 6144,
19
+ "layer_types": [
20
+ "linear_attention",
21
+ "linear_attention",
22
+ "linear_attention",
23
+ "full_attention",
24
+ "linear_attention",
25
+ "linear_attention",
26
+ "linear_attention",
27
+ "full_attention",
28
+ "linear_attention",
29
+ "linear_attention",
30
+ "linear_attention",
31
+ "full_attention",
32
+ "linear_attention",
33
+ "linear_attention",
34
+ "linear_attention",
35
+ "full_attention",
36
+ "linear_attention",
37
+ "linear_attention",
38
+ "linear_attention",
39
+ "full_attention",
40
+ "linear_attention",
41
+ "linear_attention",
42
+ "linear_attention",
43
+ "full_attention"
44
+ ],
45
+ "linear_conv_kernel_dim": 4,
46
+ "linear_key_head_dim": 128,
47
+ "linear_num_key_heads": 16,
48
+ "linear_num_value_heads": 16,
49
+ "linear_value_head_dim": 128,
50
+ "max_position_embeddings": 262144,
51
+ "mlp_only_layers": [],
52
+ "model_type": "qwen3_5_text",
53
+ "mtp_num_hidden_layers": 1,
54
+ "mtp_use_dedicated_embeddings": false,
55
+ "num_attention_heads": 8,
56
+ "num_hidden_layers": 24,
57
+ "num_key_value_heads": 2,
58
+ "rms_norm_eps": 1e-06,
59
+ "tie_word_embeddings": true,
60
+ "use_cache": true,
61
+ "vocab_size": 248320,
62
+ "mamba_ssm_dtype": "float32",
63
+ "rope_parameters": {
64
+ "mrope_interleaved": true,
65
+ "mrope_section": [
66
+ 11,
67
+ 11,
68
+ 10
69
+ ],
70
+ "rope_type": "default",
71
+ "rope_theta": 10000000,
72
+ "partial_rotary_factor": 0.25
73
+ }
74
+ },
75
+ "tie_word_embeddings": true,
76
+ "transformers_version": "4.57.0.dev0",
77
+ "video_token_id": 248057,
78
+ "vision_config": {
79
+ "deepstack_visual_indexes": [],
80
+ "depth": 24,
81
+ "hidden_act": "gelu_pytorch_tanh",
82
+ "hidden_size": 1024,
83
+ "in_channels": 3,
84
+ "initializer_range": 0.02,
85
+ "intermediate_size": 4096,
86
+ "model_type": "qwen3_5",
87
+ "num_heads": 16,
88
+ "num_position_embeddings": 2304,
89
+ "out_hidden_size": 2048,
90
+ "patch_size": 16,
91
+ "spatial_merge_size": 2,
92
+ "temporal_patch_size": 2
93
+ },
94
+ "vision_end_token_id": 248054,
95
+ "vision_start_token_id": 248053,
96
+ "mtp_num_hidden_layers": 1,
97
+ "unsloth_fixed_mtp": true
98
+ }
processor_config.json ADDED
@@ -0,0 +1,65 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "image_processor": {
3
+ "data_format": "channels_first",
4
+ "do_convert_rgb": true,
5
+ "do_normalize": true,
6
+ "do_rescale": true,
7
+ "do_resize": true,
8
+ "image_mean": [
9
+ 0.5,
10
+ 0.5,
11
+ 0.5
12
+ ],
13
+ "image_processor_type": "Qwen2VLImageProcessor",
14
+ "image_std": [
15
+ 0.5,
16
+ 0.5,
17
+ 0.5
18
+ ],
19
+ "max_pixels": 16777216,
20
+ "merge_size": 2,
21
+ "min_pixels": 65536,
22
+ "patch_size": 16,
23
+ "resample": 3,
24
+ "rescale_factor": 0.00392156862745098,
25
+ "size": {
26
+ "longest_edge": 16777216,
27
+ "shortest_edge": 65536
28
+ },
29
+ "temporal_patch_size": 2
30
+ },
31
+ "processor_class": "Qwen3VLProcessor",
32
+ "video_processor": {
33
+ "data_format": "channels_first",
34
+ "default_to_square": true,
35
+ "do_convert_rgb": true,
36
+ "do_normalize": true,
37
+ "do_rescale": true,
38
+ "do_resize": true,
39
+ "do_sample_frames": true,
40
+ "fps": 2,
41
+ "image_mean": [
42
+ 0.5,
43
+ 0.5,
44
+ 0.5
45
+ ],
46
+ "image_std": [
47
+ 0.5,
48
+ 0.5,
49
+ 0.5
50
+ ],
51
+ "max_frames": 768,
52
+ "merge_size": 2,
53
+ "min_frames": 4,
54
+ "patch_size": 16,
55
+ "resample": 3,
56
+ "rescale_factor": 0.00392156862745098,
57
+ "return_metadata": false,
58
+ "size": {
59
+ "longest_edge": 25165824,
60
+ "shortest_edge": 4096
61
+ },
62
+ "temporal_patch_size": 2,
63
+ "video_processor_type": "Qwen3VLVideoProcessor"
64
+ }
65
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:87a7830d63fcf43bf241c3c5242e96e62dd3fdc29224ca26fed8ea333db72de4
3
+ size 19989343
tokenizer_config.json ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "audio_bos_token": "<|audio_start|>",
4
+ "audio_eos_token": "<|audio_end|>",
5
+ "audio_token": "<|audio_pad|>",
6
+ "backend": "tokenizers",
7
+ "bos_token": null,
8
+ "clean_up_tokenization_spaces": false,
9
+ "eos_token": "<|im_end|>",
10
+ "errors": "replace",
11
+ "image_token": "<|image_pad|>",
12
+ "is_local": true,
13
+ "model_max_length": 262144,
14
+ "model_specific_special_tokens": {
15
+ "audio_bos_token": "<|audio_start|>",
16
+ "audio_eos_token": "<|audio_end|>",
17
+ "audio_token": "<|audio_pad|>",
18
+ "image_token": "<|image_pad|>",
19
+ "video_token": "<|video_pad|>",
20
+ "vision_bos_token": "<|vision_start|>",
21
+ "vision_eos_token": "<|vision_end|>"
22
+ },
23
+ "pad_token": "<|vision_pad|>",
24
+ "padding_side": "left",
25
+ "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
26
+ "processor_class": "Qwen3VLProcessor",
27
+ "split_special_tokens": false,
28
+ "tokenizer_class": "TokenizersBackend",
29
+ "unk_token": null,
30
+ "video_token": "<|video_pad|>",
31
+ "vision_bos_token": "<|vision_start|>",
32
+ "vision_eos_token": "<|vision_end|>"
33
+ }