Hugging Face's logo

OpenTransformer
/
unary-quantization-research

Safetensors
English
quantization
unary
thermometer-encoding
inference-engine
low-bit
Model card Files
xet
Community
OpenTransformer commited on
Commit
5e62b1a
·
verified ·
1 Parent(s): 5950fda

Add files using upload-large-folder tool

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +4 -0
  2. deepseek-r1-1.5b-gunary/model_layers_11_self_attn_v_proj_weight.sign +0 -0
  3. deepseek-r1-1.5b-gunary/model_layers_17_self_attn_v_proj_weight.gscales +0 -0
  4. deepseek-r1-1.5b-gunary/model_layers_18_self_attn_v_proj_bias.fp16 +0 -0
  5. deepseek-r1-1.5b-gunary/model_layers_1_self_attn_k_proj_weight.sign +0 -0
  6. deepseek-r1-1.5b-gunary/model_layers_22_input_layernorm_weight.fp16 +0 -0
  7. deepseek-r1-1.5b-gunary/model_layers_23_post_attention_layernorm_weight.fp16 +0 -0
  8. deepseek-r1-1.5b-gunary/model_layers_27_self_attn_q_proj_bias.fp16 +0 -0
  9. deepseek-r1-1.5b-gunary/model_layers_4_self_attn_k_proj_weight.gscales +0 -0
  10. deepseek-r1-1.5b-gunary/model_layers_8_self_attn_v_proj_weight.gscales +0 -0
  11. deepseek-r1-1.5b-gunary/model_norm_weight.fp16 +0 -0
  12. deepseek-r1-1.5b-ternary/model_layers_11_self_attn_o_proj_weight.neg +3 -0
  13. deepseek-r1-1.5b-ternary/model_layers_17_mlp_down_proj_weight.pos +3 -0
  14. deepseek-r1-1.5b-ternary/model_layers_17_mlp_gate_proj_weight.neg +3 -0
  15. deepseek-r1-1.5b-ternary/model_layers_3_mlp_up_proj_weight.neg +3 -0
  16. deepseek-r1-1.5b-unary/config.json +13 -0
  17. deepseek-r1-1.5b-unary/model_layers_0_self_attn_q_proj_weight.scales +0 -0
  18. deepseek-r1-1.5b-unary/model_layers_10_mlp_up_proj_weight.scales +0 -0
  19. deepseek-r1-1.5b-unary/model_layers_10_self_attn_q_proj_bias.fp16 +0 -0
  20. deepseek-r1-1.5b-unary/model_layers_12_self_attn_k_proj_weight.scales +0 -0
  21. deepseek-r1-1.5b-unary/model_layers_12_self_attn_o_proj_weight.scales +0 -0
  22. deepseek-r1-1.5b-unary/model_layers_14_self_attn_v_proj_weight.scales +0 -0
  23. deepseek-r1-1.5b-unary/model_layers_16_mlp_gate_proj_weight.scales +0 -0
  24. deepseek-r1-1.5b-unary/model_layers_16_self_attn_q_proj_weight.scales +0 -0
  25. deepseek-r1-1.5b-unary/model_layers_17_post_attention_layernorm_weight.fp16 +0 -0
  26. deepseek-r1-1.5b-unary/model_layers_17_self_attn_k_proj_weight.scales +0 -0
  27. deepseek-r1-1.5b-unary/model_layers_17_self_attn_q_proj_bias.fp16 +0 -0
  28. deepseek-r1-1.5b-unary/model_layers_18_self_attn_o_proj_weight.scales +0 -0
  29. deepseek-r1-1.5b-unary/model_layers_19_self_attn_k_proj_weight.sign +0 -0
  30. deepseek-r1-1.5b-unary/model_layers_1_mlp_down_proj_weight.scales +0 -0
  31. deepseek-r1-1.5b-unary/model_layers_1_self_attn_k_proj_weight.scales +0 -0
  32. deepseek-r1-1.5b-unary/model_layers_1_self_attn_v_proj_weight.sign +0 -0
  33. deepseek-r1-1.5b-unary/model_layers_20_input_layernorm_weight.fp16 +0 -0
  34. deepseek-r1-1.5b-unary/model_layers_21_self_attn_q_proj_weight.scales +0 -0
  35. deepseek-r1-1.5b-unary/model_layers_22_mlp_down_proj_weight.scales +0 -0
  36. deepseek-r1-1.5b-unary/model_layers_23_mlp_gate_proj_weight.scales +0 -0
  37. deepseek-r1-1.5b-unary/model_layers_23_self_attn_o_proj_weight.scales +0 -0
  38. deepseek-r1-1.5b-unary/model_layers_25_self_attn_q_proj_bias.fp16 +0 -0
  39. deepseek-r1-1.5b-unary/model_layers_26_self_attn_v_proj_bias.fp16 +0 -0
  40. deepseek-r1-1.5b-unary/model_layers_27_self_attn_v_proj_weight.scales +0 -0
  41. deepseek-r1-1.5b-unary/model_layers_3_self_attn_q_proj_weight.scales +0 -0
  42. deepseek-r1-1.5b-unary/model_layers_5_self_attn_k_proj_bias.fp16 +0 -0
  43. deepseek-r1-1.5b-unary/model_layers_5_self_attn_k_proj_weight.scales +0 -0
  44. deepseek-r1-1.5b-unary/model_layers_7_self_attn_k_proj_weight.scales +0 -0
  45. deepseek-r1-1.5b-unary/model_layers_8_self_attn_k_proj_weight.sign +0 -0
  46. deepseek-r1-1.5b-unary/model_layers_8_self_attn_v_proj_weight.sign +0 -0
  47. deepseek-r1-1.5b-unary/model_layers_9_self_attn_q_proj_bias.fp16 +0 -0
  48. qwen3-4b-proper-unary/model_layers_0_mlp_up_proj_weight.scales +0 -0
  49. qwen3-4b-proper-unary/model_layers_0_post_attention_layernorm_weight.fp16 +0 -0
  50. qwen3-4b-proper-unary/model_layers_0_self_attn_k_proj_weight.scales +0 -0
.gitattributes CHANGED
@@ -3263,3 +3263,7 @@ deepseek-r1-1.5b-ternary/model_layers_14_mlp_down_proj_weight.neg filter=lfs dif
3263
  deepseek-r1-1.5b-ternary/model_layers_7_mlp_gate_proj_weight.pos filter=lfs diff=lfs merge=lfs -text
3264
  deepseek-r1-1.5b-ternary/model_layers_15_mlp_up_proj_weight.neg filter=lfs diff=lfs merge=lfs -text
3265
  deepseek-r1-1.5b-ternary/model_layers_14_mlp_up_proj_weight.pos filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
3263
  deepseek-r1-1.5b-ternary/model_layers_7_mlp_gate_proj_weight.pos filter=lfs diff=lfs merge=lfs -text
3264
  deepseek-r1-1.5b-ternary/model_layers_15_mlp_up_proj_weight.neg filter=lfs diff=lfs merge=lfs -text
3265
  deepseek-r1-1.5b-ternary/model_layers_14_mlp_up_proj_weight.pos filter=lfs diff=lfs merge=lfs -text
3266
+ deepseek-r1-1.5b-ternary/model_layers_3_mlp_up_proj_weight.neg filter=lfs diff=lfs merge=lfs -text
3267
+ deepseek-r1-1.5b-ternary/model_layers_11_self_attn_o_proj_weight.neg filter=lfs diff=lfs merge=lfs -text
3268
+ deepseek-r1-1.5b-ternary/model_layers_17_mlp_down_proj_weight.pos filter=lfs diff=lfs merge=lfs -text
3269
+ deepseek-r1-1.5b-ternary/model_layers_17_mlp_gate_proj_weight.neg filter=lfs diff=lfs merge=lfs -text
deepseek-r1-1.5b-gunary/model_layers_11_self_attn_v_proj_weight.sign ADDED
Binary file (49.2 kB). View file
 
deepseek-r1-1.5b-gunary/model_layers_17_self_attn_v_proj_weight.gscales ADDED
Binary file (49.2 kB). View file
 
deepseek-r1-1.5b-gunary/model_layers_18_self_attn_v_proj_bias.fp16 ADDED
Binary file (512 Bytes). View file
 
deepseek-r1-1.5b-gunary/model_layers_1_self_attn_k_proj_weight.sign ADDED
Binary file (49.2 kB). View file
 
deepseek-r1-1.5b-gunary/model_layers_22_input_layernorm_weight.fp16 ADDED
Binary file (3.07 kB). View file
 
deepseek-r1-1.5b-gunary/model_layers_23_post_attention_layernorm_weight.fp16 ADDED
Binary file (3.07 kB). View file
 
deepseek-r1-1.5b-gunary/model_layers_27_self_attn_q_proj_bias.fp16 ADDED
Binary file (3.07 kB). View file
 
deepseek-r1-1.5b-gunary/model_layers_4_self_attn_k_proj_weight.gscales ADDED
Binary file (49.2 kB). View file
 
deepseek-r1-1.5b-gunary/model_layers_8_self_attn_v_proj_weight.gscales ADDED
Binary file (49.2 kB). View file
 
deepseek-r1-1.5b-gunary/model_norm_weight.fp16 ADDED
Binary file (3.07 kB). View file
 
deepseek-r1-1.5b-ternary/model_layers_11_self_attn_o_proj_weight.neg ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f567c0cc701f6fc315e0434b53ae7b7ba8b19cad0f46406357ef2a3d5c44b46
3
+ size 294912
deepseek-r1-1.5b-ternary/model_layers_17_mlp_down_proj_weight.pos ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0bb91223789024707b1f3c1b6ac92a9c54ea59284266a48e6960da71f54aac8
3
+ size 1720320
deepseek-r1-1.5b-ternary/model_layers_17_mlp_gate_proj_weight.neg ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e5e726100bb59e8d0c1cd9a136a25277faad2a79fe9df2a7f17f6572fd504454
3
+ size 1720320
deepseek-r1-1.5b-ternary/model_layers_3_mlp_up_proj_weight.neg ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b24be988627435effb7e84cf8b2a59d76f6ec0dfc27cb1f4e5761584434735d3
3
+ size 1720320
deepseek-r1-1.5b-unary/config.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "hidden_size": 1536,
3
+ "intermediate_size": 8960,
4
+ "num_attention_heads": 12,
5
+ "num_key_value_heads": 2,
6
+ "num_hidden_layers": 28,
7
+ "vocab_size": 151936,
8
+ "head_dim": 128,
9
+ "rope_theta": 1000000.0,
10
+ "rms_norm_eps": 1e-06,
11
+ "n_planes": 7,
12
+ "quant_type": "unary"
13
+ }
deepseek-r1-1.5b-unary/model_layers_0_self_attn_q_proj_weight.scales ADDED
Binary file (6.14 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_10_mlp_up_proj_weight.scales ADDED
Binary file (35.8 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_10_self_attn_q_proj_bias.fp16 ADDED
Binary file (3.07 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_12_self_attn_k_proj_weight.scales ADDED
Binary file (1.02 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_12_self_attn_o_proj_weight.scales ADDED
Binary file (6.14 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_14_self_attn_v_proj_weight.scales ADDED
Binary file (1.02 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_16_mlp_gate_proj_weight.scales ADDED
Binary file (35.8 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_16_self_attn_q_proj_weight.scales ADDED
Binary file (6.14 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_17_post_attention_layernorm_weight.fp16 ADDED
Binary file (3.07 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_17_self_attn_k_proj_weight.scales ADDED
Binary file (1.02 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_17_self_attn_q_proj_bias.fp16 ADDED
Binary file (3.07 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_18_self_attn_o_proj_weight.scales ADDED
Binary file (6.14 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_19_self_attn_k_proj_weight.sign ADDED
Binary file (49.2 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_1_mlp_down_proj_weight.scales ADDED
Binary file (6.14 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_1_self_attn_k_proj_weight.scales ADDED
Binary file (1.02 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_1_self_attn_v_proj_weight.sign ADDED
Binary file (49.2 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_20_input_layernorm_weight.fp16 ADDED
Binary file (3.07 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_21_self_attn_q_proj_weight.scales ADDED
Binary file (6.14 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_22_mlp_down_proj_weight.scales ADDED
Binary file (6.14 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_23_mlp_gate_proj_weight.scales ADDED
Binary file (35.8 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_23_self_attn_o_proj_weight.scales ADDED
Binary file (6.14 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_25_self_attn_q_proj_bias.fp16 ADDED
Binary file (3.07 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_26_self_attn_v_proj_bias.fp16 ADDED
Binary file (512 Bytes). View file
 
deepseek-r1-1.5b-unary/model_layers_27_self_attn_v_proj_weight.scales ADDED
Binary file (1.02 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_3_self_attn_q_proj_weight.scales ADDED
Binary file (6.14 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_5_self_attn_k_proj_bias.fp16 ADDED
Binary file (512 Bytes). View file
 
deepseek-r1-1.5b-unary/model_layers_5_self_attn_k_proj_weight.scales ADDED
Binary file (1.02 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_7_self_attn_k_proj_weight.scales ADDED
Binary file (1.02 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_8_self_attn_k_proj_weight.sign ADDED
Binary file (49.2 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_8_self_attn_v_proj_weight.sign ADDED
Binary file (49.2 kB). View file
 
deepseek-r1-1.5b-unary/model_layers_9_self_attn_q_proj_bias.fp16 ADDED
Binary file (3.07 kB). View file
 
qwen3-4b-proper-unary/model_layers_0_mlp_up_proj_weight.scales ADDED
Binary file (38.9 kB). View file
 
qwen3-4b-proper-unary/model_layers_0_post_attention_layernorm_weight.fp16 ADDED
Binary file (5.12 kB). View file
 
qwen3-4b-proper-unary/model_layers_0_self_attn_k_proj_weight.scales ADDED
Binary file (4.1 kB). View file