Hugging Face's logo

dalatexcoder
/
MiniCPM5-2B-5BitL-MLX

Text Generation
MLX
Safetensors
English
Chinese
llama
minicpm
minicpm5
long-context
tool-calling
on-device
edge-ai
conversational
4-bit precision
Model card Files
xet
 Community
dalatexcoder commited on
Commit
f33bd7b
·
verified ·
1 Parent(s): 98356ae

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +136 -0
README.md CHANGED
@@ -26,3 +26,139 @@ datasets:
26
  - openbmb/UltraData-RL-2609
27
  base_model: openbmb/MiniCPM5-2B
28
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
26
  - openbmb/UltraData-RL-2609
27
  base_model: openbmb/MiniCPM5-2B
28
  ---
29
+
30
+ Quantized from [openbmb/MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B). Effective average BPW is around 5.669.
31
+
32
+ Top 20 most-sensitive layers at 2-bit:
33
+ 1. model.layers.1.mlp.down_proj sens=1.002e+00 params=12,582,912
34
+ 2. model.layers.7.mlp.down_proj sens=4.134e-01 params=12,582,912
35
+ 3. lm_head sens=3.690e-01 params=267,386,880
36
+ 4. model.layers.1.self_attn.q_proj sens=2.767e-01 params=4,194,304
37
+ 5. model.layers.0.self_attn.v_proj sens=1.715e-01 params=524,288
38
+ 6. model.layers.41.mlp.down_proj sens=1.299e-01 params=12,582,912
39
+ 7. model.layers.1.self_attn.o_proj sens=1.231e-01 params=4,194,304
40
+ 8. model.layers.4.self_attn.v_proj sens=1.165e-01 params=524,288
41
+ 9. model.layers.2.self_attn.q_proj sens=9.487e-02 params=4,194,304
42
+ 10. model.layers.2.self_attn.v_proj sens=9.286e-02 params=524,288
43
+ 11. model.layers.0.mlp.down_proj sens=8.109e-02 params=12,582,912
44
+ 12. model.layers.13.self_attn.o_proj sens=7.899e-02 params=4,194,304
45
+ 13. model.layers.14.self_attn.o_proj sens=7.754e-02 params=4,194,304
46
+ 14. model.layers.3.self_attn.v_proj sens=7.545e-02 params=524,288
47
+ 15. model.layers.16.self_attn.o_proj sens=7.483e-02 params=4,194,304
48
+ 16. model.layers.21.self_attn.o_proj sens=7.397e-02 params=4,194,304
49
+ 17. model.layers.1.self_attn.v_proj sens=7.267e-02 params=524,288
50
+ 18. model.layers.5.self_attn.q_proj sens=6.835e-02 params=4,194,304
51
+ 19. model.layers.5.self_attn.v_proj sens=6.390e-02 params=524,288
52
+ 20. model.layers.6.mlp.down_proj sens=6.340e-02 params=12,582,912
53
+
54
+ Top 20 most-sensitive layers at 3-bit:
55
+ 1. model.layers.7.mlp.down_proj sens=1.488e-01 params=12,582,912
56
+ 2. model.layers.1.mlp.down_proj sens=8.359e-02 params=12,582,912
57
+ 3. lm_head sens=7.595e-02 params=267,386,880
58
+ 4. model.layers.2.self_attn.v_proj sens=4.404e-02 params=524,288
59
+ 5. model.layers.0.self_attn.v_proj sens=4.262e-02 params=524,288
60
+ 6. model.layers.3.self_attn.v_proj sens=3.825e-02 params=524,288
61
+ 7. model.layers.2.self_attn.q_proj sens=3.742e-02 params=4,194,304
62
+ 8. model.layers.1.self_attn.o_proj sens=3.118e-02 params=4,194,304
63
+ 9. model.layers.41.mlp.down_proj sens=2.823e-02 params=12,582,912
64
+ 10. model.layers.1.self_attn.q_proj sens=2.587e-02 params=4,194,304
65
+ 11. model.layers.5.self_attn.v_proj sens=2.548e-02 params=524,288
66
+ 12. model.layers.4.self_attn.v_proj sens=2.526e-02 params=524,288
67
+ 13. model.layers.1.self_attn.v_proj sens=2.486e-02 params=524,288
68
+ 14. model.layers.5.self_attn.q_proj sens=2.426e-02 params=4,194,304
69
+ 15. model.layers.14.self_attn.o_proj sens=2.276e-02 params=4,194,304
70
+ 16. model.layers.12.self_attn.v_proj sens=2.275e-02 params=524,288
71
+ 17. model.layers.0.mlp.down_proj sens=2.242e-02 params=12,582,912
72
+ 18. model.layers.13.self_attn.o_proj sens=2.229e-02 params=4,194,304
73
+ 19. model.layers.6.mlp.down_proj sens=2.124e-02 params=12,582,912
74
+ 20. model.layers.16.self_attn.o_proj sens=2.064e-02 params=4,194,304
75
+
76
+ Top 20 most-sensitive layers at 4-bit:
77
+ 1. model.layers.7.mlp.down_proj sens=4.436e-02 params=12,582,912
78
+ 2. model.layers.1.mlp.down_proj sens=2.787e-02 params=12,582,912
79
+ 3. model.layers.0.self_attn.v_proj sens=2.504e-02 params=524,288
80
+ 4. lm_head sens=1.819e-02 params=267,386,880
81
+ 5. model.layers.2.self_attn.v_proj sens=1.638e-02 params=524,288
82
+ 6. model.layers.5.self_attn.q_proj sens=1.346e-02 params=4,194,304
83
+ 7. model.layers.1.self_attn.o_proj sens=1.188e-02 params=4,194,304
84
+ 8. model.layers.4.self_attn.v_proj sens=1.160e-02 params=524,288
85
+ 9. model.layers.1.self_attn.q_proj sens=1.062e-02 params=4,194,304
86
+ 10. model.layers.2.self_attn.o_proj sens=1.054e-02 params=4,194,304
87
+ 11. model.layers.3.self_attn.v_proj sens=9.660e-03 params=524,288
88
+ 12. model.layers.2.self_attn.q_proj sens=9.620e-03 params=4,194,304
89
+ 13. model.layers.10.mlp.up_proj sens=9.215e-03 params=12,582,912
90
+ 14. model.layers.0.self_attn.o_proj sens=9.097e-03 params=4,194,304
91
+ 15. model.layers.1.self_attn.v_proj sens=9.091e-03 params=524,288
92
+ 16. model.layers.0.mlp.down_proj sens=8.764e-03 params=12,582,912
93
+ 17. model.layers.5.self_attn.v_proj sens=8.714e-03 params=524,288
94
+ 18. model.layers.9.self_attn.v_proj sens=8.605e-03 params=524,288
95
+ 19. model.layers.14.self_attn.o_proj sens=7.816e-03 params=4,194,304
96
+ 20. model.layers.4.mlp.up_proj sens=7.747e-03 params=12,582,912
97
+
98
+ Top 20 most-sensitive layers at 5-bit:
99
+ 1. model.layers.7.mlp.down_proj sens=1.577e-02 params=12,582,912
100
+ 2. model.layers.1.mlp.down_proj sens=9.463e-03 params=12,582,912
101
+ 3. model.layers.0.self_attn.v_proj sens=7.656e-03 params=524,288
102
+ 4. model.layers.2.self_attn.v_proj sens=6.293e-03 params=524,288
103
+ 5. model.layers.1.self_attn.o_proj sens=6.019e-03 params=4,194,304
104
+ 6. model.layers.5.mlp.up_proj sens=5.718e-03 params=12,582,912
105
+ 7. model.layers.1.self_attn.q_proj sens=5.607e-03 params=4,194,304
106
+ 8. model.layers.3.self_attn.v_proj sens=5.563e-03 params=524,288
107
+ 9. model.layers.4.mlp.down_proj sens=5.433e-03 params=12,582,912
108
+ 10. model.layers.2.mlp.gate_proj sens=5.274e-03 params=12,582,912
109
+ 11. model.layers.0.mlp.down_proj sens=5.241e-03 params=12,582,912
110
+ 12. model.layers.2.self_attn.o_proj sens=5.067e-03 params=4,194,304
111
+ 13. model.layers.2.self_attn.q_proj sens=5.060e-03 params=4,194,304
112
+ 14. model.layers.4.mlp.up_proj sens=4.983e-03 params=12,582,912
113
+ 15. model.layers.1.self_attn.v_proj sens=4.959e-03 params=524,288
114
+ 16. model.layers.0.self_attn.o_proj sens=4.954e-03 params=4,194,304
115
+ 17. model.layers.5.self_attn.v_proj sens=4.817e-03 params=524,288
116
+ 18. model.layers.0.mlp.gate_proj sens=4.768e-03 params=12,582,912
117
+ 19. lm_head sens=4.615e-03 params=267,386,880
118
+ 20. model.layers.2.mlp.up_proj sens=4.561e-03 params=12,582,912
119
+
120
+ Top 20 most-sensitive layers at 6-bit:
121
+ 1. model.layers.7.mlp.down_proj sens=6.492e-03 params=12,582,912
122
+ 2. model.layers.1.mlp.down_proj sens=6.159e-03 params=12,582,912
123
+ 3. model.layers.0.self_attn.v_proj sens=5.739e-03 params=524,288
124
+ 4. model.layers.3.self_attn.k_proj sens=5.012e-03 params=524,288
125
+ 5. model.layers.2.self_attn.v_proj sens=4.739e-03 params=524,288
126
+ 6. model.layers.0.mlp.up_proj sens=4.594e-03 params=12,582,912
127
+ 7. model.layers.2.self_attn.q_proj sens=4.409e-03 params=4,194,304
128
+ 8. model.layers.2.mlp.down_proj sens=4.399e-03 params=12,582,912
129
+ 9. model.layers.6.mlp.up_proj sens=4.348e-03 params=12,582,912
130
+ 10. model.layers.1.self_attn.v_proj sens=4.302e-03 params=524,288
131
+ 11. model.layers.2.self_attn.o_proj sens=4.259e-03 params=4,194,304
132
+ 12. model.layers.4.self_attn.q_proj sens=4.223e-03 params=4,194,304
133
+ 13. model.layers.1.mlp.gate_proj sens=4.110e-03 params=12,582,912
134
+ 14. model.layers.2.self_attn.k_proj sens=4.094e-03 params=524,288
135
+ 15. model.layers.6.self_attn.q_proj sens=3.977e-03 params=4,194,304
136
+ 16. model.layers.6.mlp.gate_proj sens=3.973e-03 params=12,582,912
137
+ 17. model.layers.1.self_attn.o_proj sens=3.940e-03 params=4,194,304
138
+ 18. model.layers.4.mlp.down_proj sens=3.883e-03 params=12,582,912
139
+ 19. model.layers.5.mlp.gate_proj sens=3.870e-03 params=12,582,912
140
+ 20. model.layers.0.self_attn.o_proj sens=3.824e-03 params=4,194,304
141
+
142
+ Top 20 most-sensitive layers at 8-bit:
143
+ 1. model.layers.1.self_attn.v_proj sens=4.394e-03 params=524,288
144
+ 2. model.layers.2.self_attn.v_proj sens=4.088e-03 params=524,288
145
+ 3. model.layers.4.self_attn.k_proj sens=3.958e-03 params=524,288
146
+ 4. model.layers.1.mlp.gate_proj sens=3.900e-03 params=12,582,912
147
+ 5. model.layers.0.mlp.up_proj sens=3.824e-03 params=12,582,912
148
+ 6. model.layers.0.mlp.down_proj sens=3.799e-03 params=12,582,912
149
+ 7. model.layers.0.self_attn.k_proj sens=3.778e-03 params=524,288
150
+ 8. model.layers.0.self_attn.o_proj sens=3.697e-03 params=4,194,304
151
+ 9. model.layers.4.mlp.up_proj sens=3.644e-03 params=12,582,912
152
+ 10. model.layers.0.self_attn.v_proj sens=3.592e-03 params=524,288
153
+ 11. model.layers.4.mlp.gate_proj sens=3.513e-03 params=12,582,912
154
+ 12. model.layers.5.self_attn.v_proj sens=3.422e-03 params=524,288
155
+ 13. model.layers.1.mlp.up_proj sens=3.421e-03 params=12,582,912
156
+ 14. model.layers.3.self_attn.q_proj sens=3.398e-03 params=4,194,304
157
+ 15. model.layers.6.mlp.gate_proj sens=3.394e-03 params=12,582,912
158
+ 16. model.layers.4.mlp.down_proj sens=3.328e-03 params=12,582,912
159
+ 17. model.layers.5.self_attn.q_proj sens=3.321e-03 params=4,194,304
160
+ 18. model.layers.0.self_attn.q_proj sens=3.279e-03 params=4,194,304
161
+ 19. model.layers.2.mlp.gate_proj sens=3.275e-03 params=12,582,912
162
+ 20. model.layers.0.mlp.gate_proj sens=3.252e-03 params=12,582,912
163
+
164
+ Optimization result: 34 @ 3-bit, 95 @ 4-bit, 84 @ 5-bit, 42 @ 6-bit, 40 @ 8-bit