File size: 13,649 Bytes
98356ae f33bd7b 8be25e0 f33bd7b 8be25e0 f33bd7b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 | ---
license: apache-2.0
language:
- en
- zh
library_name: mlx
pipeline_tag: text-generation
tags:
- minicpm
- minicpm5
- llama
- text-generation
- long-context
- tool-calling
- on-device
- edge-ai
- mlx
datasets:
- openbmb/Ultra-FineWeb
- openbmb/UltraX-Preview
- openbmb/Ultra-FineWeb-L3
- openbmb/UltraData-Math
- openbmb/UltraData-Code
- openbmb/UltraData-SFT-2605
- openbmb/UltraData-SFT-Agent-2609
- openbmb/UltraData-RL-2609
base_model: openbmb/MiniCPM5-2B
---
Quantized from [openbmb/MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B). Effective average BPW is around 5.669.
Top 20 most-sensitive layers at 2-bit:
1. model.layers.1.mlp.down_proj sens=1.002e+00 params=12,582,912
2. model.layers.7.mlp.down_proj sens=4.134e-01 params=12,582,912
3. lm_head sens=3.690e-01 params=267,386,880
4. model.layers.1.self_attn.q_proj sens=2.767e-01 params=4,194,304
5. model.layers.0.self_attn.v_proj sens=1.715e-01 params=524,288
6. model.layers.41.mlp.down_proj sens=1.299e-01 params=12,582,912
7. model.layers.1.self_attn.o_proj sens=1.231e-01 params=4,194,304
8. model.layers.4.self_attn.v_proj sens=1.165e-01 params=524,288
9. model.layers.2.self_attn.q_proj sens=9.487e-02 params=4,194,304
10. model.layers.2.self_attn.v_proj sens=9.286e-02 params=524,288
11. model.layers.0.mlp.down_proj sens=8.109e-02 params=12,582,912
12. model.layers.13.self_attn.o_proj sens=7.899e-02 params=4,194,304
13. model.layers.14.self_attn.o_proj sens=7.754e-02 params=4,194,304
14. model.layers.3.self_attn.v_proj sens=7.545e-02 params=524,288
15. model.layers.16.self_attn.o_proj sens=7.483e-02 params=4,194,304
16. model.layers.21.self_attn.o_proj sens=7.397e-02 params=4,194,304
17. model.layers.1.self_attn.v_proj sens=7.267e-02 params=524,288
18. model.layers.5.self_attn.q_proj sens=6.835e-02 params=4,194,304
19. model.layers.5.self_attn.v_proj sens=6.390e-02 params=524,288
20. model.layers.6.mlp.down_proj sens=6.340e-02 params=12,582,912
Top 20 most-sensitive layers at 3-bit:
1. model.layers.7.mlp.down_proj sens=1.488e-01 params=12,582,912
2. model.layers.1.mlp.down_proj sens=8.359e-02 params=12,582,912
3. lm_head sens=7.595e-02 params=267,386,880
4. model.layers.2.self_attn.v_proj sens=4.404e-02 params=524,288
5. model.layers.0.self_attn.v_proj sens=4.262e-02 params=524,288
6. model.layers.3.self_attn.v_proj sens=3.825e-02 params=524,288
7. model.layers.2.self_attn.q_proj sens=3.742e-02 params=4,194,304
8. model.layers.1.self_attn.o_proj sens=3.118e-02 params=4,194,304
9. model.layers.41.mlp.down_proj sens=2.823e-02 params=12,582,912
10. model.layers.1.self_attn.q_proj sens=2.587e-02 params=4,194,304
11. model.layers.5.self_attn.v_proj sens=2.548e-02 params=524,288
12. model.layers.4.self_attn.v_proj sens=2.526e-02 params=524,288
13. model.layers.1.self_attn.v_proj sens=2.486e-02 params=524,288
14. model.layers.5.self_attn.q_proj sens=2.426e-02 params=4,194,304
15. model.layers.14.self_attn.o_proj sens=2.276e-02 params=4,194,304
16. model.layers.12.self_attn.v_proj sens=2.275e-02 params=524,288
17. model.layers.0.mlp.down_proj sens=2.242e-02 params=12,582,912
18. model.layers.13.self_attn.o_proj sens=2.229e-02 params=4,194,304
19. model.layers.6.mlp.down_proj sens=2.124e-02 params=12,582,912
20. model.layers.16.self_attn.o_proj sens=2.064e-02 params=4,194,304
Top 20 most-sensitive layers at 4-bit:
1. model.layers.7.mlp.down_proj sens=4.436e-02 params=12,582,912
2. model.layers.1.mlp.down_proj sens=2.787e-02 params=12,582,912
3. model.layers.0.self_attn.v_proj sens=2.504e-02 params=524,288
4. lm_head sens=1.819e-02 params=267,386,880
5. model.layers.2.self_attn.v_proj sens=1.638e-02 params=524,288
6. model.layers.5.self_attn.q_proj sens=1.346e-02 params=4,194,304
7. model.layers.1.self_attn.o_proj sens=1.188e-02 params=4,194,304
8. model.layers.4.self_attn.v_proj sens=1.160e-02 params=524,288
9. model.layers.1.self_attn.q_proj sens=1.062e-02 params=4,194,304
10. model.layers.2.self_attn.o_proj sens=1.054e-02 params=4,194,304
11. model.layers.3.self_attn.v_proj sens=9.660e-03 params=524,288
12. model.layers.2.self_attn.q_proj sens=9.620e-03 params=4,194,304
13. model.layers.10.mlp.up_proj sens=9.215e-03 params=12,582,912
14. model.layers.0.self_attn.o_proj sens=9.097e-03 params=4,194,304
15. model.layers.1.self_attn.v_proj sens=9.091e-03 params=524,288
16. model.layers.0.mlp.down_proj sens=8.764e-03 params=12,582,912
17. model.layers.5.self_attn.v_proj sens=8.714e-03 params=524,288
18. model.layers.9.self_attn.v_proj sens=8.605e-03 params=524,288
19. model.layers.14.self_attn.o_proj sens=7.816e-03 params=4,194,304
20. model.layers.4.mlp.up_proj sens=7.747e-03 params=12,582,912
Top 20 most-sensitive layers at 5-bit:
1. model.layers.7.mlp.down_proj sens=1.577e-02 params=12,582,912
2. model.layers.1.mlp.down_proj sens=9.463e-03 params=12,582,912
3. model.layers.0.self_attn.v_proj sens=7.656e-03 params=524,288
4. model.layers.2.self_attn.v_proj sens=6.293e-03 params=524,288
5. model.layers.1.self_attn.o_proj sens=6.019e-03 params=4,194,304
6. model.layers.5.mlp.up_proj sens=5.718e-03 params=12,582,912
7. model.layers.1.self_attn.q_proj sens=5.607e-03 params=4,194,304
8. model.layers.3.self_attn.v_proj sens=5.563e-03 params=524,288
9. model.layers.4.mlp.down_proj sens=5.433e-03 params=12,582,912
10. model.layers.2.mlp.gate_proj sens=5.274e-03 params=12,582,912
11. model.layers.0.mlp.down_proj sens=5.241e-03 params=12,582,912
12. model.layers.2.self_attn.o_proj sens=5.067e-03 params=4,194,304
13. model.layers.2.self_attn.q_proj sens=5.060e-03 params=4,194,304
14. model.layers.4.mlp.up_proj sens=4.983e-03 params=12,582,912
15. model.layers.1.self_attn.v_proj sens=4.959e-03 params=524,288
16. model.layers.0.self_attn.o_proj sens=4.954e-03 params=4,194,304
17. model.layers.5.self_attn.v_proj sens=4.817e-03 params=524,288
18. model.layers.0.mlp.gate_proj sens=4.768e-03 params=12,582,912
19. lm_head sens=4.615e-03 params=267,386,880
20. model.layers.2.mlp.up_proj sens=4.561e-03 params=12,582,912
Top 20 most-sensitive layers at 6-bit:
1. model.layers.7.mlp.down_proj sens=6.492e-03 params=12,582,912
2. model.layers.1.mlp.down_proj sens=6.159e-03 params=12,582,912
3. model.layers.0.self_attn.v_proj sens=5.739e-03 params=524,288
4. model.layers.3.self_attn.k_proj sens=5.012e-03 params=524,288
5. model.layers.2.self_attn.v_proj sens=4.739e-03 params=524,288
6. model.layers.0.mlp.up_proj sens=4.594e-03 params=12,582,912
7. model.layers.2.self_attn.q_proj sens=4.409e-03 params=4,194,304
8. model.layers.2.mlp.down_proj sens=4.399e-03 params=12,582,912
9. model.layers.6.mlp.up_proj sens=4.348e-03 params=12,582,912
10. model.layers.1.self_attn.v_proj sens=4.302e-03 params=524,288
11. model.layers.2.self_attn.o_proj sens=4.259e-03 params=4,194,304
12. model.layers.4.self_attn.q_proj sens=4.223e-03 params=4,194,304
13. model.layers.1.mlp.gate_proj sens=4.110e-03 params=12,582,912
14. model.layers.2.self_attn.k_proj sens=4.094e-03 params=524,288
15. model.layers.6.self_attn.q_proj sens=3.977e-03 params=4,194,304
16. model.layers.6.mlp.gate_proj sens=3.973e-03 params=12,582,912
17. model.layers.1.self_attn.o_proj sens=3.940e-03 params=4,194,304
18. model.layers.4.mlp.down_proj sens=3.883e-03 params=12,582,912
19. model.layers.5.mlp.gate_proj sens=3.870e-03 params=12,582,912
20. model.layers.0.self_attn.o_proj sens=3.824e-03 params=4,194,304
Top 20 most-sensitive layers at 8-bit:
1. model.layers.1.self_attn.v_proj sens=4.394e-03 params=524,288
2. model.layers.2.self_attn.v_proj sens=4.088e-03 params=524,288
3. model.layers.4.self_attn.k_proj sens=3.958e-03 params=524,288
4. model.layers.1.mlp.gate_proj sens=3.900e-03 params=12,582,912
5. model.layers.0.mlp.up_proj sens=3.824e-03 params=12,582,912
6. model.layers.0.mlp.down_proj sens=3.799e-03 params=12,582,912
7. model.layers.0.self_attn.k_proj sens=3.778e-03 params=524,288
8. model.layers.0.self_attn.o_proj sens=3.697e-03 params=4,194,304
9. model.layers.4.mlp.up_proj sens=3.644e-03 params=12,582,912
10. model.layers.0.self_attn.v_proj sens=3.592e-03 params=524,288
11. model.layers.4.mlp.gate_proj sens=3.513e-03 params=12,582,912
12. model.layers.5.self_attn.v_proj sens=3.422e-03 params=524,288
13. model.layers.1.mlp.up_proj sens=3.421e-03 params=12,582,912
14. model.layers.3.self_attn.q_proj sens=3.398e-03 params=4,194,304
15. model.layers.6.mlp.gate_proj sens=3.394e-03 params=12,582,912
16. model.layers.4.mlp.down_proj sens=3.328e-03 params=12,582,912
17. model.layers.5.self_attn.q_proj sens=3.321e-03 params=4,194,304
18. model.layers.0.self_attn.q_proj sens=3.279e-03 params=4,194,304
19. model.layers.2.mlp.gate_proj sens=3.275e-03 params=12,582,912
20. model.layers.0.mlp.gate_proj sens=3.252e-03 params=12,582,912
Optimization result: 34 @ 3-bit, 95 @ 4-bit, 84 @ 5-bit, 42 @ 6-bit, 40 @ 8-bit |