Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -24,22 +24,22 @@ tags:
|
|
| 24 |
|
| 25 |
| Parameter | Value |
|
| 26 |
| :-------- | :---: |
|
| 27 |
-
| **direction_index** | 19.
|
| 28 |
-
| **attn.o_proj.max_weight** | 1.
|
| 29 |
-
| **attn.o_proj.max_weight_position** |
|
| 30 |
-
| **attn.o_proj.min_weight** | 0.
|
| 31 |
-
| **attn.o_proj.min_weight_distance** |
|
| 32 |
-
| **mlp.down_proj.max_weight** | 1.
|
| 33 |
-
| **mlp.down_proj.max_weight_position** |
|
| 34 |
-
| **mlp.down_proj.min_weight** | 0.
|
| 35 |
-
| **mlp.down_proj.min_weight_distance** |
|
| 36 |
|
| 37 |
## Performance
|
| 38 |
|
| 39 |
| Metric | This model | Original model ([Qwen/Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct)) |
|
| 40 |
| :----- | :--------: | :---------------------------: |
|
| 41 |
-
| **KL divergence** | 0.
|
| 42 |
-
| **Refusals** |
|
| 43 |
|
| 44 |
-----
|
| 45 |
|
|
|
|
| 24 |
|
| 25 |
| Parameter | Value |
|
| 26 |
| :-------- | :---: |
|
| 27 |
+
| **direction_index** | 19.57 |
|
| 28 |
+
| **attn.o_proj.max_weight** | 1.01 |
|
| 29 |
+
| **attn.o_proj.max_weight_position** | 26.72 |
|
| 30 |
+
| **attn.o_proj.min_weight** | 0.77 |
|
| 31 |
+
| **attn.o_proj.min_weight_distance** | 15.16 |
|
| 32 |
+
| **mlp.down_proj.max_weight** | 1.35 |
|
| 33 |
+
| **mlp.down_proj.max_weight_position** | 17.74 |
|
| 34 |
+
| **mlp.down_proj.min_weight** | 0.16 |
|
| 35 |
+
| **mlp.down_proj.min_weight_distance** | 13.48 |
|
| 36 |
|
| 37 |
## Performance
|
| 38 |
|
| 39 |
| Metric | This model | Original model ([Qwen/Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct)) |
|
| 40 |
| :----- | :--------: | :---------------------------: |
|
| 41 |
+
| **KL divergence** | 0.0298 | 0 *(by definition)* |
|
| 42 |
+
| **Refusals** | 3/100 | 95/100 |
|
| 43 |
|
| 44 |
-----
|
| 45 |
|