Hugging Face's logo

nvidia
/
GLM-5.3-Flash-FP4-Local

Image-Text-to-Text
Safetensors
Model Optimizer
glm5_next
nvidia
ModelOpt
GLM-5
quantized
FP4
fp4
conversational
8-bit precision
modelopt
Model card Files
xet
 Community
6
nv-lukea commited on
Commit
f5a0bab
·
verified ·
1 Parent(s): f83de1b

Update checkpoint weights and quantization configuration

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. config.json +0 -0
  2. hf_quant_config.json +2973 -0
  3. tensors/model-00001-of-00036.safetensors → model-00001-of-00040.safetensors +2 -2
  4. tensors/model-00002-of-00036.safetensors → model-00002-of-00040.safetensors +2 -2
  5. tensors/model-00003-of-00036.safetensors → model-00003-of-00040.safetensors +2 -2
  6. tensors/model-00004-of-00036.safetensors → model-00004-of-00040.safetensors +2 -2
  7. model-00005-of-00040.safetensors +3 -0
  8. model-00006-of-00040.safetensors +3 -0
  9. model-00007-of-00040.safetensors +3 -0
  10. model-00008-of-00040.safetensors +3 -0
  11. model-00009-of-00040.safetensors +3 -0
  12. model-00010-of-00040.safetensors +3 -0
  13. model-00011-of-00040.safetensors +3 -0
  14. model-00012-of-00040.safetensors +3 -0
  15. model-00013-of-00040.safetensors +3 -0
  16. model-00014-of-00040.safetensors +3 -0
  17. model-00015-of-00040.safetensors +3 -0
  18. model-00016-of-00040.safetensors +3 -0
  19. model-00017-of-00040.safetensors +3 -0
  20. model-00018-of-00040.safetensors +3 -0
  21. model-00019-of-00040.safetensors +3 -0
  22. model-00020-of-00040.safetensors +3 -0
  23. model-00021-of-00040.safetensors +3 -0
  24. model-00022-of-00040.safetensors +3 -0
  25. model-00023-of-00040.safetensors +3 -0
  26. model-00024-of-00040.safetensors +3 -0
  27. model-00025-of-00040.safetensors +3 -0
  28. model-00026-of-00040.safetensors +3 -0
  29. model-00027-of-00040.safetensors +3 -0
  30. model-00028-of-00040.safetensors +3 -0
  31. model-00029-of-00040.safetensors +3 -0
  32. model-00030-of-00040.safetensors +3 -0
  33. model-00031-of-00040.safetensors +3 -0
  34. model-00032-of-00040.safetensors +3 -0
  35. model-00033-of-00040.safetensors +3 -0
  36. model-00034-of-00040.safetensors +3 -0
  37. model-00035-of-00040.safetensors +3 -0
  38. model-00036-of-00040.safetensors +3 -0
  39. model-00037-of-00040.safetensors +3 -0
  40. model-00038-of-00040.safetensors +3 -0
  41. model-00039-of-00040.safetensors +3 -0
  42. model-00040-of-00040.safetensors +3 -0
  43. model-inputscales.safetensors +3 -0
  44. model.safetensors.index.json +2 -2
  45. tensors/model-00005-of-00036.safetensors +0 -3
  46. tensors/model-00006-of-00036.safetensors +0 -3
  47. tensors/model-00007-of-00036.safetensors +0 -3
  48. tensors/model-00008-of-00036.safetensors +0 -3
  49. tensors/model-00009-of-00036.safetensors +0 -3
  50. tensors/model-00010-of-00036.safetensors +0 -3
config.json CHANGED
The diff for this file is too large to render. See raw diff
 
hf_quant_config.json ADDED
@@ -0,0 +1,2973 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "exclude_modules": [],
3
+ "quant_algo": "MIXED_PRECISION",
4
+ "quant_method": "modelopt",
5
+ "quantized_layers": {
6
+ "lm_head": {
7
+ "quant_algo": "MXFP8",
8
+ "weight_block_size": [
9
+ 1,
10
+ 32
11
+ ]
12
+ },
13
+ "model.language_model.layers.0.mlp.down_proj": {
14
+ "quant_algo": "MXFP8",
15
+ "weight_block_size": [
16
+ 1,
17
+ 32
18
+ ]
19
+ },
20
+ "model.language_model.layers.0.mlp.gate_proj": {
21
+ "quant_algo": "MXFP8",
22
+ "weight_block_size": [
23
+ 1,
24
+ 32
25
+ ]
26
+ },
27
+ "model.language_model.layers.0.mlp.up_proj": {
28
+ "quant_algo": "MXFP8",
29
+ "weight_block_size": [
30
+ 1,
31
+ 32
32
+ ]
33
+ },
34
+ "model.language_model.layers.0.self_attn.k_proj": {
35
+ "quant_algo": "MXFP8",
36
+ "weight_block_size": [
37
+ 1,
38
+ 32
39
+ ]
40
+ },
41
+ "model.language_model.layers.0.self_attn.o_proj": {
42
+ "quant_algo": "MXFP8",
43
+ "weight_block_size": [
44
+ 1,
45
+ 32
46
+ ]
47
+ },
48
+ "model.language_model.layers.0.self_attn.q_proj": {
49
+ "quant_algo": "MXFP8",
50
+ "weight_block_size": [
51
+ 1,
52
+ 32
53
+ ]
54
+ },
55
+ "model.language_model.layers.0.self_attn.v_proj": {
56
+ "quant_algo": "MXFP8",
57
+ "weight_block_size": [
58
+ 1,
59
+ 32
60
+ ]
61
+ },
62
+ "model.language_model.layers.1.mlp.down_proj": {
63
+ "quant_algo": "MXFP8",
64
+ "weight_block_size": [
65
+ 1,
66
+ 32
67
+ ]
68
+ },
69
+ "model.language_model.layers.1.mlp.gate_proj": {
70
+ "quant_algo": "MXFP8",
71
+ "weight_block_size": [
72
+ 1,
73
+ 32
74
+ ]
75
+ },
76
+ "model.language_model.layers.1.mlp.up_proj": {
77
+ "quant_algo": "MXFP8",
78
+ "weight_block_size": [
79
+ 1,
80
+ 32
81
+ ]
82
+ },
83
+ "model.language_model.layers.1.self_attn.k_proj": {
84
+ "quant_algo": "MXFP8",
85
+ "weight_block_size": [
86
+ 1,
87
+ 32
88
+ ]
89
+ },
90
+ "model.language_model.layers.1.self_attn.o_proj": {
91
+ "quant_algo": "MXFP8",
92
+ "weight_block_size": [
93
+ 1,
94
+ 32
95
+ ]
96
+ },
97
+ "model.language_model.layers.1.self_attn.q_proj": {
98
+ "quant_algo": "MXFP8",
99
+ "weight_block_size": [
100
+ 1,
101
+ 32
102
+ ]
103
+ },
104
+ "model.language_model.layers.1.self_attn.v_proj": {
105
+ "quant_algo": "MXFP8",
106
+ "weight_block_size": [
107
+ 1,
108
+ 32
109
+ ]
110
+ },
111
+ "model.language_model.layers.10.mlp.experts": {
112
+ "group_size": 16,
113
+ "quant_algo": "NVFP4",
114
+ "weight_scale_encoding": "csf_32x2_v1"
115
+ },
116
+ "model.language_model.layers.10.mlp.shared_experts.down_proj": {
117
+ "quant_algo": "FP8_PB_WO",
118
+ "weight_block_size": [
119
+ 128,
120
+ 128
121
+ ]
122
+ },
123
+ "model.language_model.layers.10.mlp.shared_experts.gate_proj": {
124
+ "quant_algo": "FP8_PB_WO",
125
+ "weight_block_size": [
126
+ 128,
127
+ 128
128
+ ]
129
+ },
130
+ "model.language_model.layers.10.mlp.shared_experts.up_proj": {
131
+ "quant_algo": "FP8_PB_WO",
132
+ "weight_block_size": [
133
+ 128,
134
+ 128
135
+ ]
136
+ },
137
+ "model.language_model.layers.10.self_attn.k_proj": {
138
+ "quant_algo": "MXFP8",
139
+ "weight_block_size": [
140
+ 1,
141
+ 32
142
+ ]
143
+ },
144
+ "model.language_model.layers.10.self_attn.o_proj": {
145
+ "quant_algo": "MXFP8",
146
+ "weight_block_size": [
147
+ 1,
148
+ 32
149
+ ]
150
+ },
151
+ "model.language_model.layers.10.self_attn.q_proj": {
152
+ "quant_algo": "MXFP8",
153
+ "weight_block_size": [
154
+ 1,
155
+ 32
156
+ ]
157
+ },
158
+ "model.language_model.layers.10.self_attn.v_proj": {
159
+ "quant_algo": "MXFP8",
160
+ "weight_block_size": [
161
+ 1,
162
+ 32
163
+ ]
164
+ },
165
+ "model.language_model.layers.11.mlp.experts": {
166
+ "group_size": 16,
167
+ "quant_algo": "NVFP4",
168
+ "weight_scale_encoding": "csf_32x2_v1"
169
+ },
170
+ "model.language_model.layers.11.mlp.shared_experts.down_proj": {
171
+ "quant_algo": "FP8_PB_WO",
172
+ "weight_block_size": [
173
+ 128,
174
+ 128
175
+ ]
176
+ },
177
+ "model.language_model.layers.11.mlp.shared_experts.gate_proj": {
178
+ "quant_algo": "FP8_PB_WO",
179
+ "weight_block_size": [
180
+ 128,
181
+ 128
182
+ ]
183
+ },
184
+ "model.language_model.layers.11.mlp.shared_experts.up_proj": {
185
+ "quant_algo": "FP8_PB_WO",
186
+ "weight_block_size": [
187
+ 128,
188
+ 128
189
+ ]
190
+ },
191
+ "model.language_model.layers.11.self_attn.kv_a_proj_with_mqa": {
192
+ "quant_algo": "FP8_PB_WO",
193
+ "weight_block_size": [
194
+ 128,
195
+ 128
196
+ ]
197
+ },
198
+ "model.language_model.layers.11.self_attn.o_proj": {
199
+ "quant_algo": "FP8_PB_WO",
200
+ "weight_block_size": [
201
+ 128,
202
+ 128
203
+ ]
204
+ },
205
+ "model.language_model.layers.11.self_attn.q_a_proj": {
206
+ "quant_algo": "FP8_PB_WO",
207
+ "weight_block_size": [
208
+ 128,
209
+ 128
210
+ ]
211
+ },
212
+ "model.language_model.layers.11.self_attn.q_b_proj": {
213
+ "quant_algo": "FP8_PB_WO",
214
+ "weight_block_size": [
215
+ 128,
216
+ 128
217
+ ]
218
+ },
219
+ "model.language_model.layers.12.mlp.experts": {
220
+ "group_size": 16,
221
+ "quant_algo": "NVFP4",
222
+ "weight_scale_encoding": "csf_32x2_v1"
223
+ },
224
+ "model.language_model.layers.12.mlp.shared_experts.down_proj": {
225
+ "quant_algo": "FP8_PB_WO",
226
+ "weight_block_size": [
227
+ 128,
228
+ 128
229
+ ]
230
+ },
231
+ "model.language_model.layers.12.mlp.shared_experts.gate_proj": {
232
+ "quant_algo": "FP8_PB_WO",
233
+ "weight_block_size": [
234
+ 128,
235
+ 128
236
+ ]
237
+ },
238
+ "model.language_model.layers.12.mlp.shared_experts.up_proj": {
239
+ "quant_algo": "FP8_PB_WO",
240
+ "weight_block_size": [
241
+ 128,
242
+ 128
243
+ ]
244
+ },
245
+ "model.language_model.layers.12.self_attn.k_proj": {
246
+ "quant_algo": "MXFP8",
247
+ "weight_block_size": [
248
+ 1,
249
+ 32
250
+ ]
251
+ },
252
+ "model.language_model.layers.12.self_attn.o_proj": {
253
+ "quant_algo": "MXFP8",
254
+ "weight_block_size": [
255
+ 1,
256
+ 32
257
+ ]
258
+ },
259
+ "model.language_model.layers.12.self_attn.q_proj": {
260
+ "quant_algo": "MXFP8",
261
+ "weight_block_size": [
262
+ 1,
263
+ 32
264
+ ]
265
+ },
266
+ "model.language_model.layers.12.self_attn.v_proj": {
267
+ "quant_algo": "MXFP8",
268
+ "weight_block_size": [
269
+ 1,
270
+ 32
271
+ ]
272
+ },
273
+ "model.language_model.layers.13.mlp.experts": {
274
+ "group_size": 16,
275
+ "quant_algo": "NVFP4",
276
+ "weight_scale_encoding": "csf_32x2_v1"
277
+ },
278
+ "model.language_model.layers.13.mlp.shared_experts.down_proj": {
279
+ "quant_algo": "FP8_PB_WO",
280
+ "weight_block_size": [
281
+ 128,
282
+ 128
283
+ ]
284
+ },
285
+ "model.language_model.layers.13.mlp.shared_experts.gate_proj": {
286
+ "quant_algo": "FP8_PB_WO",
287
+ "weight_block_size": [
288
+ 128,
289
+ 128
290
+ ]
291
+ },
292
+ "model.language_model.layers.13.mlp.shared_experts.up_proj": {
293
+ "quant_algo": "FP8_PB_WO",
294
+ "weight_block_size": [
295
+ 128,
296
+ 128
297
+ ]
298
+ },
299
+ "model.language_model.layers.13.self_attn.k_proj": {
300
+ "quant_algo": "MXFP8",
301
+ "weight_block_size": [
302
+ 1,
303
+ 32
304
+ ]
305
+ },
306
+ "model.language_model.layers.13.self_attn.o_proj": {
307
+ "quant_algo": "MXFP8",
308
+ "weight_block_size": [
309
+ 1,
310
+ 32
311
+ ]
312
+ },
313
+ "model.language_model.layers.13.self_attn.q_proj": {
314
+ "quant_algo": "MXFP8",
315
+ "weight_block_size": [
316
+ 1,
317
+ 32
318
+ ]
319
+ },
320
+ "model.language_model.layers.13.self_attn.v_proj": {
321
+ "quant_algo": "MXFP8",
322
+ "weight_block_size": [
323
+ 1,
324
+ 32
325
+ ]
326
+ },
327
+ "model.language_model.layers.14.mlp.experts": {
328
+ "group_size": 16,
329
+ "quant_algo": "NVFP4",
330
+ "weight_scale_encoding": "csf_32x2_v1"
331
+ },
332
+ "model.language_model.layers.14.mlp.shared_experts.down_proj": {
333
+ "quant_algo": "FP8_PB_WO",
334
+ "weight_block_size": [
335
+ 128,
336
+ 128
337
+ ]
338
+ },
339
+ "model.language_model.layers.14.mlp.shared_experts.gate_proj": {
340
+ "quant_algo": "FP8_PB_WO",
341
+ "weight_block_size": [
342
+ 128,
343
+ 128
344
+ ]
345
+ },
346
+ "model.language_model.layers.14.mlp.shared_experts.up_proj": {
347
+ "quant_algo": "FP8_PB_WO",
348
+ "weight_block_size": [
349
+ 128,
350
+ 128
351
+ ]
352
+ },
353
+ "model.language_model.layers.14.self_attn.k_proj": {
354
+ "quant_algo": "MXFP8",
355
+ "weight_block_size": [
356
+ 1,
357
+ 32
358
+ ]
359
+ },
360
+ "model.language_model.layers.14.self_attn.o_proj": {
361
+ "quant_algo": "MXFP8",
362
+ "weight_block_size": [
363
+ 1,
364
+ 32
365
+ ]
366
+ },
367
+ "model.language_model.layers.14.self_attn.q_proj": {
368
+ "quant_algo": "MXFP8",
369
+ "weight_block_size": [
370
+ 1,
371
+ 32
372
+ ]
373
+ },
374
+ "model.language_model.layers.14.self_attn.v_proj": {
375
+ "quant_algo": "MXFP8",
376
+ "weight_block_size": [
377
+ 1,
378
+ 32
379
+ ]
380
+ },
381
+ "model.language_model.layers.15.mlp.experts": {
382
+ "group_size": 16,
383
+ "quant_algo": "NVFP4",
384
+ "weight_scale_encoding": "csf_32x2_v1"
385
+ },
386
+ "model.language_model.layers.15.mlp.shared_experts.down_proj": {
387
+ "quant_algo": "FP8_PB_WO",
388
+ "weight_block_size": [
389
+ 128,
390
+ 128
391
+ ]
392
+ },
393
+ "model.language_model.layers.15.mlp.shared_experts.gate_proj": {
394
+ "quant_algo": "FP8_PB_WO",
395
+ "weight_block_size": [
396
+ 128,
397
+ 128
398
+ ]
399
+ },
400
+ "model.language_model.layers.15.mlp.shared_experts.up_proj": {
401
+ "quant_algo": "FP8_PB_WO",
402
+ "weight_block_size": [
403
+ 128,
404
+ 128
405
+ ]
406
+ },
407
+ "model.language_model.layers.15.self_attn.kv_a_proj_with_mqa": {
408
+ "quant_algo": "FP8_PB_WO",
409
+ "weight_block_size": [
410
+ 128,
411
+ 128
412
+ ]
413
+ },
414
+ "model.language_model.layers.15.self_attn.o_proj": {
415
+ "quant_algo": "FP8_PB_WO",
416
+ "weight_block_size": [
417
+ 128,
418
+ 128
419
+ ]
420
+ },
421
+ "model.language_model.layers.15.self_attn.q_a_proj": {
422
+ "quant_algo": "FP8_PB_WO",
423
+ "weight_block_size": [
424
+ 128,
425
+ 128
426
+ ]
427
+ },
428
+ "model.language_model.layers.15.self_attn.q_b_proj": {
429
+ "quant_algo": "FP8_PB_WO",
430
+ "weight_block_size": [
431
+ 128,
432
+ 128
433
+ ]
434
+ },
435
+ "model.language_model.layers.16.mlp.experts": {
436
+ "group_size": 16,
437
+ "quant_algo": "NVFP4",
438
+ "weight_scale_encoding": "csf_32x2_v1"
439
+ },
440
+ "model.language_model.layers.16.mlp.shared_experts.down_proj": {
441
+ "quant_algo": "FP8_PB_WO",
442
+ "weight_block_size": [
443
+ 128,
444
+ 128
445
+ ]
446
+ },
447
+ "model.language_model.layers.16.mlp.shared_experts.gate_proj": {
448
+ "quant_algo": "FP8_PB_WO",
449
+ "weight_block_size": [
450
+ 128,
451
+ 128
452
+ ]
453
+ },
454
+ "model.language_model.layers.16.mlp.shared_experts.up_proj": {
455
+ "quant_algo": "FP8_PB_WO",
456
+ "weight_block_size": [
457
+ 128,
458
+ 128
459
+ ]
460
+ },
461
+ "model.language_model.layers.16.self_attn.k_proj": {
462
+ "quant_algo": "MXFP8",
463
+ "weight_block_size": [
464
+ 1,
465
+ 32
466
+ ]
467
+ },
468
+ "model.language_model.layers.16.self_attn.o_proj": {
469
+ "quant_algo": "MXFP8",
470
+ "weight_block_size": [
471
+ 1,
472
+ 32
473
+ ]
474
+ },
475
+ "model.language_model.layers.16.self_attn.q_proj": {
476
+ "quant_algo": "MXFP8",
477
+ "weight_block_size": [
478
+ 1,
479
+ 32
480
+ ]
481
+ },
482
+ "model.language_model.layers.16.self_attn.v_proj": {
483
+ "quant_algo": "MXFP8",
484
+ "weight_block_size": [
485
+ 1,
486
+ 32
487
+ ]
488
+ },
489
+ "model.language_model.layers.17.mlp.experts": {
490
+ "group_size": 16,
491
+ "quant_algo": "NVFP4",
492
+ "weight_scale_encoding": "csf_32x2_v1"
493
+ },
494
+ "model.language_model.layers.17.mlp.shared_experts.down_proj": {
495
+ "quant_algo": "FP8_PB_WO",
496
+ "weight_block_size": [
497
+ 128,
498
+ 128
499
+ ]
500
+ },
501
+ "model.language_model.layers.17.mlp.shared_experts.gate_proj": {
502
+ "quant_algo": "FP8_PB_WO",
503
+ "weight_block_size": [
504
+ 128,
505
+ 128
506
+ ]
507
+ },
508
+ "model.language_model.layers.17.mlp.shared_experts.up_proj": {
509
+ "quant_algo": "FP8_PB_WO",
510
+ "weight_block_size": [
511
+ 128,
512
+ 128
513
+ ]
514
+ },
515
+ "model.language_model.layers.17.self_attn.k_proj": {
516
+ "quant_algo": "MXFP8",
517
+ "weight_block_size": [
518
+ 1,
519
+ 32
520
+ ]
521
+ },
522
+ "model.language_model.layers.17.self_attn.o_proj": {
523
+ "quant_algo": "MXFP8",
524
+ "weight_block_size": [
525
+ 1,
526
+ 32
527
+ ]
528
+ },
529
+ "model.language_model.layers.17.self_attn.q_proj": {
530
+ "quant_algo": "MXFP8",
531
+ "weight_block_size": [
532
+ 1,
533
+ 32
534
+ ]
535
+ },
536
+ "model.language_model.layers.17.self_attn.v_proj": {
537
+ "quant_algo": "MXFP8",
538
+ "weight_block_size": [
539
+ 1,
540
+ 32
541
+ ]
542
+ },
543
+ "model.language_model.layers.18.mlp.experts": {
544
+ "group_size": 16,
545
+ "quant_algo": "NVFP4",
546
+ "weight_scale_encoding": "csf_32x2_v1"
547
+ },
548
+ "model.language_model.layers.18.mlp.shared_experts.down_proj": {
549
+ "quant_algo": "FP8_PB_WO",
550
+ "weight_block_size": [
551
+ 128,
552
+ 128
553
+ ]
554
+ },
555
+ "model.language_model.layers.18.mlp.shared_experts.gate_proj": {
556
+ "quant_algo": "FP8_PB_WO",
557
+ "weight_block_size": [
558
+ 128,
559
+ 128
560
+ ]
561
+ },
562
+ "model.language_model.layers.18.mlp.shared_experts.up_proj": {
563
+ "quant_algo": "FP8_PB_WO",
564
+ "weight_block_size": [
565
+ 128,
566
+ 128
567
+ ]
568
+ },
569
+ "model.language_model.layers.18.self_attn.k_proj": {
570
+ "quant_algo": "MXFP8",
571
+ "weight_block_size": [
572
+ 1,
573
+ 32
574
+ ]
575
+ },
576
+ "model.language_model.layers.18.self_attn.o_proj": {
577
+ "quant_algo": "MXFP8",
578
+ "weight_block_size": [
579
+ 1,
580
+ 32
581
+ ]
582
+ },
583
+ "model.language_model.layers.18.self_attn.q_proj": {
584
+ "quant_algo": "MXFP8",
585
+ "weight_block_size": [
586
+ 1,
587
+ 32
588
+ ]
589
+ },
590
+ "model.language_model.layers.18.self_attn.v_proj": {
591
+ "quant_algo": "MXFP8",
592
+ "weight_block_size": [
593
+ 1,
594
+ 32
595
+ ]
596
+ },
597
+ "model.language_model.layers.19.mlp.experts": {
598
+ "group_size": 16,
599
+ "quant_algo": "NVFP4",
600
+ "weight_scale_encoding": "csf_32x2_v1"
601
+ },
602
+ "model.language_model.layers.19.mlp.shared_experts.down_proj": {
603
+ "quant_algo": "FP8_PB_WO",
604
+ "weight_block_size": [
605
+ 128,
606
+ 128
607
+ ]
608
+ },
609
+ "model.language_model.layers.19.mlp.shared_experts.gate_proj": {
610
+ "quant_algo": "FP8_PB_WO",
611
+ "weight_block_size": [
612
+ 128,
613
+ 128
614
+ ]
615
+ },
616
+ "model.language_model.layers.19.mlp.shared_experts.up_proj": {
617
+ "quant_algo": "FP8_PB_WO",
618
+ "weight_block_size": [
619
+ 128,
620
+ 128
621
+ ]
622
+ },
623
+ "model.language_model.layers.19.self_attn.kv_a_proj_with_mqa": {
624
+ "quant_algo": "FP8_PB_WO",
625
+ "weight_block_size": [
626
+ 128,
627
+ 128
628
+ ]
629
+ },
630
+ "model.language_model.layers.19.self_attn.o_proj": {
631
+ "quant_algo": "FP8_PB_WO",
632
+ "weight_block_size": [
633
+ 128,
634
+ 128
635
+ ]
636
+ },
637
+ "model.language_model.layers.19.self_attn.q_a_proj": {
638
+ "quant_algo": "FP8_PB_WO",
639
+ "weight_block_size": [
640
+ 128,
641
+ 128
642
+ ]
643
+ },
644
+ "model.language_model.layers.19.self_attn.q_b_proj": {
645
+ "quant_algo": "FP8_PB_WO",
646
+ "weight_block_size": [
647
+ 128,
648
+ 128
649
+ ]
650
+ },
651
+ "model.language_model.layers.2.mlp.down_proj": {
652
+ "quant_algo": "MXFP8",
653
+ "weight_block_size": [
654
+ 1,
655
+ 32
656
+ ]
657
+ },
658
+ "model.language_model.layers.2.mlp.gate_proj": {
659
+ "quant_algo": "MXFP8",
660
+ "weight_block_size": [
661
+ 1,
662
+ 32
663
+ ]
664
+ },
665
+ "model.language_model.layers.2.mlp.up_proj": {
666
+ "quant_algo": "MXFP8",
667
+ "weight_block_size": [
668
+ 1,
669
+ 32
670
+ ]
671
+ },
672
+ "model.language_model.layers.2.self_attn.k_proj": {
673
+ "quant_algo": "MXFP8",
674
+ "weight_block_size": [
675
+ 1,
676
+ 32
677
+ ]
678
+ },
679
+ "model.language_model.layers.2.self_attn.o_proj": {
680
+ "quant_algo": "MXFP8",
681
+ "weight_block_size": [
682
+ 1,
683
+ 32
684
+ ]
685
+ },
686
+ "model.language_model.layers.2.self_attn.q_proj": {
687
+ "quant_algo": "MXFP8",
688
+ "weight_block_size": [
689
+ 1,
690
+ 32
691
+ ]
692
+ },
693
+ "model.language_model.layers.2.self_attn.v_proj": {
694
+ "quant_algo": "MXFP8",
695
+ "weight_block_size": [
696
+ 1,
697
+ 32
698
+ ]
699
+ },
700
+ "model.language_model.layers.20.mlp.experts": {
701
+ "group_size": 16,
702
+ "quant_algo": "NVFP4",
703
+ "weight_scale_encoding": "csf_32x2_v1"
704
+ },
705
+ "model.language_model.layers.20.mlp.shared_experts.down_proj": {
706
+ "quant_algo": "FP8_PB_WO",
707
+ "weight_block_size": [
708
+ 128,
709
+ 128
710
+ ]
711
+ },
712
+ "model.language_model.layers.20.mlp.shared_experts.gate_proj": {
713
+ "quant_algo": "FP8_PB_WO",
714
+ "weight_block_size": [
715
+ 128,
716
+ 128
717
+ ]
718
+ },
719
+ "model.language_model.layers.20.mlp.shared_experts.up_proj": {
720
+ "quant_algo": "FP8_PB_WO",
721
+ "weight_block_size": [
722
+ 128,
723
+ 128
724
+ ]
725
+ },
726
+ "model.language_model.layers.20.self_attn.k_proj": {
727
+ "quant_algo": "MXFP8",
728
+ "weight_block_size": [
729
+ 1,
730
+ 32
731
+ ]
732
+ },
733
+ "model.language_model.layers.20.self_attn.o_proj": {
734
+ "quant_algo": "MXFP8",
735
+ "weight_block_size": [
736
+ 1,
737
+ 32
738
+ ]
739
+ },
740
+ "model.language_model.layers.20.self_attn.q_proj": {
741
+ "quant_algo": "MXFP8",
742
+ "weight_block_size": [
743
+ 1,
744
+ 32
745
+ ]
746
+ },
747
+ "model.language_model.layers.20.self_attn.v_proj": {
748
+ "quant_algo": "MXFP8",
749
+ "weight_block_size": [
750
+ 1,
751
+ 32
752
+ ]
753
+ },
754
+ "model.language_model.layers.21.mlp.experts": {
755
+ "group_size": 16,
756
+ "quant_algo": "NVFP4",
757
+ "weight_scale_encoding": "csf_32x2_v1"
758
+ },
759
+ "model.language_model.layers.21.mlp.shared_experts.down_proj": {
760
+ "quant_algo": "FP8_PB_WO",
761
+ "weight_block_size": [
762
+ 128,
763
+ 128
764
+ ]
765
+ },
766
+ "model.language_model.layers.21.mlp.shared_experts.gate_proj": {
767
+ "quant_algo": "FP8_PB_WO",
768
+ "weight_block_size": [
769
+ 128,
770
+ 128
771
+ ]
772
+ },
773
+ "model.language_model.layers.21.mlp.shared_experts.up_proj": {
774
+ "quant_algo": "FP8_PB_WO",
775
+ "weight_block_size": [
776
+ 128,
777
+ 128
778
+ ]
779
+ },
780
+ "model.language_model.layers.21.self_attn.k_proj": {
781
+ "quant_algo": "MXFP8",
782
+ "weight_block_size": [
783
+ 1,
784
+ 32
785
+ ]
786
+ },
787
+ "model.language_model.layers.21.self_attn.o_proj": {
788
+ "quant_algo": "MXFP8",
789
+ "weight_block_size": [
790
+ 1,
791
+ 32
792
+ ]
793
+ },
794
+ "model.language_model.layers.21.self_attn.q_proj": {
795
+ "quant_algo": "MXFP8",
796
+ "weight_block_size": [
797
+ 1,
798
+ 32
799
+ ]
800
+ },
801
+ "model.language_model.layers.21.self_attn.v_proj": {
802
+ "quant_algo": "MXFP8",
803
+ "weight_block_size": [
804
+ 1,
805
+ 32
806
+ ]
807
+ },
808
+ "model.language_model.layers.22.mlp.experts": {
809
+ "group_size": 16,
810
+ "quant_algo": "NVFP4",
811
+ "weight_scale_encoding": "csf_32x2_v1"
812
+ },
813
+ "model.language_model.layers.22.mlp.shared_experts.down_proj": {
814
+ "quant_algo": "FP8_PB_WO",
815
+ "weight_block_size": [
816
+ 128,
817
+ 128
818
+ ]
819
+ },
820
+ "model.language_model.layers.22.mlp.shared_experts.gate_proj": {
821
+ "quant_algo": "FP8_PB_WO",
822
+ "weight_block_size": [
823
+ 128,
824
+ 128
825
+ ]
826
+ },
827
+ "model.language_model.layers.22.mlp.shared_experts.up_proj": {
828
+ "quant_algo": "FP8_PB_WO",
829
+ "weight_block_size": [
830
+ 128,
831
+ 128
832
+ ]
833
+ },
834
+ "model.language_model.layers.22.self_attn.k_proj": {
835
+ "quant_algo": "MXFP8",
836
+ "weight_block_size": [
837
+ 1,
838
+ 32
839
+ ]
840
+ },
841
+ "model.language_model.layers.22.self_attn.o_proj": {
842
+ "quant_algo": "MXFP8",
843
+ "weight_block_size": [
844
+ 1,
845
+ 32
846
+ ]
847
+ },
848
+ "model.language_model.layers.22.self_attn.q_proj": {
849
+ "quant_algo": "MXFP8",
850
+ "weight_block_size": [
851
+ 1,
852
+ 32
853
+ ]
854
+ },
855
+ "model.language_model.layers.22.self_attn.v_proj": {
856
+ "quant_algo": "MXFP8",
857
+ "weight_block_size": [
858
+ 1,
859
+ 32
860
+ ]
861
+ },
862
+ "model.language_model.layers.23.mlp.experts": {
863
+ "group_size": 16,
864
+ "quant_algo": "NVFP4",
865
+ "weight_scale_encoding": "csf_32x2_v1"
866
+ },
867
+ "model.language_model.layers.23.mlp.shared_experts.down_proj": {
868
+ "quant_algo": "FP8_PB_WO",
869
+ "weight_block_size": [
870
+ 128,
871
+ 128
872
+ ]
873
+ },
874
+ "model.language_model.layers.23.mlp.shared_experts.gate_proj": {
875
+ "quant_algo": "FP8_PB_WO",
876
+ "weight_block_size": [
877
+ 128,
878
+ 128
879
+ ]
880
+ },
881
+ "model.language_model.layers.23.mlp.shared_experts.up_proj": {
882
+ "quant_algo": "FP8_PB_WO",
883
+ "weight_block_size": [
884
+ 128,
885
+ 128
886
+ ]
887
+ },
888
+ "model.language_model.layers.23.self_attn.kv_a_proj_with_mqa": {
889
+ "quant_algo": "FP8_PB_WO",
890
+ "weight_block_size": [
891
+ 128,
892
+ 128
893
+ ]
894
+ },
895
+ "model.language_model.layers.23.self_attn.o_proj": {
896
+ "quant_algo": "FP8_PB_WO",
897
+ "weight_block_size": [
898
+ 128,
899
+ 128
900
+ ]
901
+ },
902
+ "model.language_model.layers.23.self_attn.q_a_proj": {
903
+ "quant_algo": "FP8_PB_WO",
904
+ "weight_block_size": [
905
+ 128,
906
+ 128
907
+ ]
908
+ },
909
+ "model.language_model.layers.23.self_attn.q_b_proj": {
910
+ "quant_algo": "FP8_PB_WO",
911
+ "weight_block_size": [
912
+ 128,
913
+ 128
914
+ ]
915
+ },
916
+ "model.language_model.layers.24.mlp.experts": {
917
+ "group_size": 16,
918
+ "quant_algo": "NVFP4",
919
+ "weight_scale_encoding": "csf_32x2_v1"
920
+ },
921
+ "model.language_model.layers.24.mlp.shared_experts.down_proj": {
922
+ "quant_algo": "FP8_PB_WO",
923
+ "weight_block_size": [
924
+ 128,
925
+ 128
926
+ ]
927
+ },
928
+ "model.language_model.layers.24.mlp.shared_experts.gate_proj": {
929
+ "quant_algo": "FP8_PB_WO",
930
+ "weight_block_size": [
931
+ 128,
932
+ 128
933
+ ]
934
+ },
935
+ "model.language_model.layers.24.mlp.shared_experts.up_proj": {
936
+ "quant_algo": "FP8_PB_WO",
937
+ "weight_block_size": [
938
+ 128,
939
+ 128
940
+ ]
941
+ },
942
+ "model.language_model.layers.24.self_attn.k_proj": {
943
+ "quant_algo": "MXFP8",
944
+ "weight_block_size": [
945
+ 1,
946
+ 32
947
+ ]
948
+ },
949
+ "model.language_model.layers.24.self_attn.o_proj": {
950
+ "quant_algo": "MXFP8",
951
+ "weight_block_size": [
952
+ 1,
953
+ 32
954
+ ]
955
+ },
956
+ "model.language_model.layers.24.self_attn.q_proj": {
957
+ "quant_algo": "MXFP8",
958
+ "weight_block_size": [
959
+ 1,
960
+ 32
961
+ ]
962
+ },
963
+ "model.language_model.layers.24.self_attn.v_proj": {
964
+ "quant_algo": "MXFP8",
965
+ "weight_block_size": [
966
+ 1,
967
+ 32
968
+ ]
969
+ },
970
+ "model.language_model.layers.25.mlp.experts": {
971
+ "group_size": 16,
972
+ "quant_algo": "NVFP4",
973
+ "weight_scale_encoding": "csf_32x2_v1"
974
+ },
975
+ "model.language_model.layers.25.mlp.shared_experts.down_proj": {
976
+ "quant_algo": "FP8_PB_WO",
977
+ "weight_block_size": [
978
+ 128,
979
+ 128
980
+ ]
981
+ },
982
+ "model.language_model.layers.25.mlp.shared_experts.gate_proj": {
983
+ "quant_algo": "FP8_PB_WO",
984
+ "weight_block_size": [
985
+ 128,
986
+ 128
987
+ ]
988
+ },
989
+ "model.language_model.layers.25.mlp.shared_experts.up_proj": {
990
+ "quant_algo": "FP8_PB_WO",
991
+ "weight_block_size": [
992
+ 128,
993
+ 128
994
+ ]
995
+ },
996
+ "model.language_model.layers.25.self_attn.k_proj": {
997
+ "quant_algo": "MXFP8",
998
+ "weight_block_size": [
999
+ 1,
1000
+ 32
1001
+ ]
1002
+ },
1003
+ "model.language_model.layers.25.self_attn.o_proj": {
1004
+ "quant_algo": "MXFP8",
1005
+ "weight_block_size": [
1006
+ 1,
1007
+ 32
1008
+ ]
1009
+ },
1010
+ "model.language_model.layers.25.self_attn.q_proj": {
1011
+ "quant_algo": "MXFP8",
1012
+ "weight_block_size": [
1013
+ 1,
1014
+ 32
1015
+ ]
1016
+ },
1017
+ "model.language_model.layers.25.self_attn.v_proj": {
1018
+ "quant_algo": "MXFP8",
1019
+ "weight_block_size": [
1020
+ 1,
1021
+ 32
1022
+ ]
1023
+ },
1024
+ "model.language_model.layers.26.mlp.experts": {
1025
+ "group_size": 16,
1026
+ "quant_algo": "NVFP4",
1027
+ "weight_scale_encoding": "csf_32x2_v1"
1028
+ },
1029
+ "model.language_model.layers.26.mlp.shared_experts.down_proj": {
1030
+ "quant_algo": "FP8_PB_WO",
1031
+ "weight_block_size": [
1032
+ 128,
1033
+ 128
1034
+ ]
1035
+ },
1036
+ "model.language_model.layers.26.mlp.shared_experts.gate_proj": {
1037
+ "quant_algo": "FP8_PB_WO",
1038
+ "weight_block_size": [
1039
+ 128,
1040
+ 128
1041
+ ]
1042
+ },
1043
+ "model.language_model.layers.26.mlp.shared_experts.up_proj": {
1044
+ "quant_algo": "FP8_PB_WO",
1045
+ "weight_block_size": [
1046
+ 128,
1047
+ 128
1048
+ ]
1049
+ },
1050
+ "model.language_model.layers.26.self_attn.k_proj": {
1051
+ "quant_algo": "MXFP8",
1052
+ "weight_block_size": [
1053
+ 1,
1054
+ 32
1055
+ ]
1056
+ },
1057
+ "model.language_model.layers.26.self_attn.o_proj": {
1058
+ "quant_algo": "MXFP8",
1059
+ "weight_block_size": [
1060
+ 1,
1061
+ 32
1062
+ ]
1063
+ },
1064
+ "model.language_model.layers.26.self_attn.q_proj": {
1065
+ "quant_algo": "MXFP8",
1066
+ "weight_block_size": [
1067
+ 1,
1068
+ 32
1069
+ ]
1070
+ },
1071
+ "model.language_model.layers.26.self_attn.v_proj": {
1072
+ "quant_algo": "MXFP8",
1073
+ "weight_block_size": [
1074
+ 1,
1075
+ 32
1076
+ ]
1077
+ },
1078
+ "model.language_model.layers.27.mlp.experts": {
1079
+ "group_size": 16,
1080
+ "quant_algo": "NVFP4",
1081
+ "weight_scale_encoding": "csf_32x2_v1"
1082
+ },
1083
+ "model.language_model.layers.27.mlp.shared_experts.down_proj": {
1084
+ "quant_algo": "FP8_PB_WO",
1085
+ "weight_block_size": [
1086
+ 128,
1087
+ 128
1088
+ ]
1089
+ },
1090
+ "model.language_model.layers.27.mlp.shared_experts.gate_proj": {
1091
+ "quant_algo": "FP8_PB_WO",
1092
+ "weight_block_size": [
1093
+ 128,
1094
+ 128
1095
+ ]
1096
+ },
1097
+ "model.language_model.layers.27.mlp.shared_experts.up_proj": {
1098
+ "quant_algo": "FP8_PB_WO",
1099
+ "weight_block_size": [
1100
+ 128,
1101
+ 128
1102
+ ]
1103
+ },
1104
+ "model.language_model.layers.27.self_attn.kv_a_proj_with_mqa": {
1105
+ "quant_algo": "FP8_PB_WO",
1106
+ "weight_block_size": [
1107
+ 128,
1108
+ 128
1109
+ ]
1110
+ },
1111
+ "model.language_model.layers.27.self_attn.o_proj": {
1112
+ "quant_algo": "FP8_PB_WO",
1113
+ "weight_block_size": [
1114
+ 128,
1115
+ 128
1116
+ ]
1117
+ },
1118
+ "model.language_model.layers.27.self_attn.q_a_proj": {
1119
+ "quant_algo": "FP8_PB_WO",
1120
+ "weight_block_size": [
1121
+ 128,
1122
+ 128
1123
+ ]
1124
+ },
1125
+ "model.language_model.layers.27.self_attn.q_b_proj": {
1126
+ "quant_algo": "FP8_PB_WO",
1127
+ "weight_block_size": [
1128
+ 128,
1129
+ 128
1130
+ ]
1131
+ },
1132
+ "model.language_model.layers.28.mlp.experts": {
1133
+ "group_size": 16,
1134
+ "quant_algo": "NVFP4",
1135
+ "weight_scale_encoding": "csf_32x2_v1"
1136
+ },
1137
+ "model.language_model.layers.28.mlp.shared_experts.down_proj": {
1138
+ "quant_algo": "FP8_PB_WO",
1139
+ "weight_block_size": [
1140
+ 128,
1141
+ 128
1142
+ ]
1143
+ },
1144
+ "model.language_model.layers.28.mlp.shared_experts.gate_proj": {
1145
+ "quant_algo": "FP8_PB_WO",
1146
+ "weight_block_size": [
1147
+ 128,
1148
+ 128
1149
+ ]
1150
+ },
1151
+ "model.language_model.layers.28.mlp.shared_experts.up_proj": {
1152
+ "quant_algo": "FP8_PB_WO",
1153
+ "weight_block_size": [
1154
+ 128,
1155
+ 128
1156
+ ]
1157
+ },
1158
+ "model.language_model.layers.28.self_attn.k_proj": {
1159
+ "quant_algo": "MXFP8",
1160
+ "weight_block_size": [
1161
+ 1,
1162
+ 32
1163
+ ]
1164
+ },
1165
+ "model.language_model.layers.28.self_attn.o_proj": {
1166
+ "quant_algo": "MXFP8",
1167
+ "weight_block_size": [
1168
+ 1,
1169
+ 32
1170
+ ]
1171
+ },
1172
+ "model.language_model.layers.28.self_attn.q_proj": {
1173
+ "quant_algo": "MXFP8",
1174
+ "weight_block_size": [
1175
+ 1,
1176
+ 32
1177
+ ]
1178
+ },
1179
+ "model.language_model.layers.28.self_attn.v_proj": {
1180
+ "quant_algo": "MXFP8",
1181
+ "weight_block_size": [
1182
+ 1,
1183
+ 32
1184
+ ]
1185
+ },
1186
+ "model.language_model.layers.29.mlp.experts": {
1187
+ "group_size": 16,
1188
+ "quant_algo": "NVFP4",
1189
+ "weight_scale_encoding": "csf_32x2_v1"
1190
+ },
1191
+ "model.language_model.layers.29.mlp.shared_experts.down_proj": {
1192
+ "quant_algo": "FP8_PB_WO",
1193
+ "weight_block_size": [
1194
+ 128,
1195
+ 128
1196
+ ]
1197
+ },
1198
+ "model.language_model.layers.29.mlp.shared_experts.gate_proj": {
1199
+ "quant_algo": "FP8_PB_WO",
1200
+ "weight_block_size": [
1201
+ 128,
1202
+ 128
1203
+ ]
1204
+ },
1205
+ "model.language_model.layers.29.mlp.shared_experts.up_proj": {
1206
+ "quant_algo": "FP8_PB_WO",
1207
+ "weight_block_size": [
1208
+ 128,
1209
+ 128
1210
+ ]
1211
+ },
1212
+ "model.language_model.layers.29.self_attn.k_proj": {
1213
+ "quant_algo": "MXFP8",
1214
+ "weight_block_size": [
1215
+ 1,
1216
+ 32
1217
+ ]
1218
+ },
1219
+ "model.language_model.layers.29.self_attn.o_proj": {
1220
+ "quant_algo": "MXFP8",
1221
+ "weight_block_size": [
1222
+ 1,
1223
+ 32
1224
+ ]
1225
+ },
1226
+ "model.language_model.layers.29.self_attn.q_proj": {
1227
+ "quant_algo": "MXFP8",
1228
+ "weight_block_size": [
1229
+ 1,
1230
+ 32
1231
+ ]
1232
+ },
1233
+ "model.language_model.layers.29.self_attn.v_proj": {
1234
+ "quant_algo": "MXFP8",
1235
+ "weight_block_size": [
1236
+ 1,
1237
+ 32
1238
+ ]
1239
+ },
1240
+ "model.language_model.layers.3.mlp.experts": {
1241
+ "group_size": 16,
1242
+ "quant_algo": "NVFP4",
1243
+ "weight_scale_encoding": "csf_32x2_v1"
1244
+ },
1245
+ "model.language_model.layers.3.mlp.shared_experts.down_proj": {
1246
+ "quant_algo": "FP8_PB_WO",
1247
+ "weight_block_size": [
1248
+ 128,
1249
+ 128
1250
+ ]
1251
+ },
1252
+ "model.language_model.layers.3.mlp.shared_experts.gate_proj": {
1253
+ "quant_algo": "FP8_PB_WO",
1254
+ "weight_block_size": [
1255
+ 128,
1256
+ 128
1257
+ ]
1258
+ },
1259
+ "model.language_model.layers.3.mlp.shared_experts.up_proj": {
1260
+ "quant_algo": "FP8_PB_WO",
1261
+ "weight_block_size": [
1262
+ 128,
1263
+ 128
1264
+ ]
1265
+ },
1266
+ "model.language_model.layers.3.self_attn.kv_a_proj_with_mqa": {
1267
+ "quant_algo": "FP8_PB_WO",
1268
+ "weight_block_size": [
1269
+ 128,
1270
+ 128
1271
+ ]
1272
+ },
1273
+ "model.language_model.layers.3.self_attn.o_proj": {
1274
+ "quant_algo": "FP8_PB_WO",
1275
+ "weight_block_size": [
1276
+ 128,
1277
+ 128
1278
+ ]
1279
+ },
1280
+ "model.language_model.layers.3.self_attn.q_a_proj": {
1281
+ "quant_algo": "FP8_PB_WO",
1282
+ "weight_block_size": [
1283
+ 128,
1284
+ 128
1285
+ ]
1286
+ },
1287
+ "model.language_model.layers.3.self_attn.q_b_proj": {
1288
+ "quant_algo": "FP8_PB_WO",
1289
+ "weight_block_size": [
1290
+ 128,
1291
+ 128
1292
+ ]
1293
+ },
1294
+ "model.language_model.layers.30.mlp.experts": {
1295
+ "group_size": 16,
1296
+ "quant_algo": "NVFP4",
1297
+ "weight_scale_encoding": "csf_32x2_v1"
1298
+ },
1299
+ "model.language_model.layers.30.mlp.shared_experts.down_proj": {
1300
+ "quant_algo": "FP8_PB_WO",
1301
+ "weight_block_size": [
1302
+ 128,
1303
+ 128
1304
+ ]
1305
+ },
1306
+ "model.language_model.layers.30.mlp.shared_experts.gate_proj": {
1307
+ "quant_algo": "FP8_PB_WO",
1308
+ "weight_block_size": [
1309
+ 128,
1310
+ 128
1311
+ ]
1312
+ },
1313
+ "model.language_model.layers.30.mlp.shared_experts.up_proj": {
1314
+ "quant_algo": "FP8_PB_WO",
1315
+ "weight_block_size": [
1316
+ 128,
1317
+ 128
1318
+ ]
1319
+ },
1320
+ "model.language_model.layers.30.self_attn.k_proj": {
1321
+ "quant_algo": "MXFP8",
1322
+ "weight_block_size": [
1323
+ 1,
1324
+ 32
1325
+ ]
1326
+ },
1327
+ "model.language_model.layers.30.self_attn.o_proj": {
1328
+ "quant_algo": "MXFP8",
1329
+ "weight_block_size": [
1330
+ 1,
1331
+ 32
1332
+ ]
1333
+ },
1334
+ "model.language_model.layers.30.self_attn.q_proj": {
1335
+ "quant_algo": "MXFP8",
1336
+ "weight_block_size": [
1337
+ 1,
1338
+ 32
1339
+ ]
1340
+ },
1341
+ "model.language_model.layers.30.self_attn.v_proj": {
1342
+ "quant_algo": "MXFP8",
1343
+ "weight_block_size": [
1344
+ 1,
1345
+ 32
1346
+ ]
1347
+ },
1348
+ "model.language_model.layers.31.mlp.experts": {
1349
+ "group_size": 16,
1350
+ "quant_algo": "NVFP4",
1351
+ "weight_scale_encoding": "csf_32x2_v1"
1352
+ },
1353
+ "model.language_model.layers.31.mlp.shared_experts.down_proj": {
1354
+ "quant_algo": "FP8_PB_WO",
1355
+ "weight_block_size": [
1356
+ 128,
1357
+ 128
1358
+ ]
1359
+ },
1360
+ "model.language_model.layers.31.mlp.shared_experts.gate_proj": {
1361
+ "quant_algo": "FP8_PB_WO",
1362
+ "weight_block_size": [
1363
+ 128,
1364
+ 128
1365
+ ]
1366
+ },
1367
+ "model.language_model.layers.31.mlp.shared_experts.up_proj": {
1368
+ "quant_algo": "FP8_PB_WO",
1369
+ "weight_block_size": [
1370
+ 128,
1371
+ 128
1372
+ ]
1373
+ },
1374
+ "model.language_model.layers.31.self_attn.kv_a_proj_with_mqa": {
1375
+ "quant_algo": "FP8_PB_WO",
1376
+ "weight_block_size": [
1377
+ 128,
1378
+ 128
1379
+ ]
1380
+ },
1381
+ "model.language_model.layers.31.self_attn.o_proj": {
1382
+ "quant_algo": "FP8_PB_WO",
1383
+ "weight_block_size": [
1384
+ 128,
1385
+ 128
1386
+ ]
1387
+ },
1388
+ "model.language_model.layers.31.self_attn.q_a_proj": {
1389
+ "quant_algo": "FP8_PB_WO",
1390
+ "weight_block_size": [
1391
+ 128,
1392
+ 128
1393
+ ]
1394
+ },
1395
+ "model.language_model.layers.31.self_attn.q_b_proj": {
1396
+ "quant_algo": "FP8_PB_WO",
1397
+ "weight_block_size": [
1398
+ 128,
1399
+ 128
1400
+ ]
1401
+ },
1402
+ "model.language_model.layers.32.mlp.experts": {
1403
+ "group_size": 16,
1404
+ "quant_algo": "NVFP4",
1405
+ "weight_scale_encoding": "csf_32x2_v1"
1406
+ },
1407
+ "model.language_model.layers.32.mlp.shared_experts.down_proj": {
1408
+ "quant_algo": "FP8_PB_WO",
1409
+ "weight_block_size": [
1410
+ 128,
1411
+ 128
1412
+ ]
1413
+ },
1414
+ "model.language_model.layers.32.mlp.shared_experts.gate_proj": {
1415
+ "quant_algo": "FP8_PB_WO",
1416
+ "weight_block_size": [
1417
+ 128,
1418
+ 128
1419
+ ]
1420
+ },
1421
+ "model.language_model.layers.32.mlp.shared_experts.up_proj": {
1422
+ "quant_algo": "FP8_PB_WO",
1423
+ "weight_block_size": [
1424
+ 128,
1425
+ 128
1426
+ ]
1427
+ },
1428
+ "model.language_model.layers.32.self_attn.k_proj": {
1429
+ "quant_algo": "MXFP8",
1430
+ "weight_block_size": [
1431
+ 1,
1432
+ 32
1433
+ ]
1434
+ },
1435
+ "model.language_model.layers.32.self_attn.o_proj": {
1436
+ "quant_algo": "MXFP8",
1437
+ "weight_block_size": [
1438
+ 1,
1439
+ 32
1440
+ ]
1441
+ },
1442
+ "model.language_model.layers.32.self_attn.q_proj": {
1443
+ "quant_algo": "MXFP8",
1444
+ "weight_block_size": [
1445
+ 1,
1446
+ 32
1447
+ ]
1448
+ },
1449
+ "model.language_model.layers.32.self_attn.v_proj": {
1450
+ "quant_algo": "MXFP8",
1451
+ "weight_block_size": [
1452
+ 1,
1453
+ 32
1454
+ ]
1455
+ },
1456
+ "model.language_model.layers.33.mlp.experts": {
1457
+ "group_size": 16,
1458
+ "quant_algo": "NVFP4",
1459
+ "weight_scale_encoding": "csf_32x2_v1"
1460
+ },
1461
+ "model.language_model.layers.33.mlp.shared_experts.down_proj": {
1462
+ "quant_algo": "FP8_PB_WO",
1463
+ "weight_block_size": [
1464
+ 128,
1465
+ 128
1466
+ ]
1467
+ },
1468
+ "model.language_model.layers.33.mlp.shared_experts.gate_proj": {
1469
+ "quant_algo": "FP8_PB_WO",
1470
+ "weight_block_size": [
1471
+ 128,
1472
+ 128
1473
+ ]
1474
+ },
1475
+ "model.language_model.layers.33.mlp.shared_experts.up_proj": {
1476
+ "quant_algo": "FP8_PB_WO",
1477
+ "weight_block_size": [
1478
+ 128,
1479
+ 128
1480
+ ]
1481
+ },
1482
+ "model.language_model.layers.33.self_attn.k_proj": {
1483
+ "quant_algo": "MXFP8",
1484
+ "weight_block_size": [
1485
+ 1,
1486
+ 32
1487
+ ]
1488
+ },
1489
+ "model.language_model.layers.33.self_attn.o_proj": {
1490
+ "quant_algo": "MXFP8",
1491
+ "weight_block_size": [
1492
+ 1,
1493
+ 32
1494
+ ]
1495
+ },
1496
+ "model.language_model.layers.33.self_attn.q_proj": {
1497
+ "quant_algo": "MXFP8",
1498
+ "weight_block_size": [
1499
+ 1,
1500
+ 32
1501
+ ]
1502
+ },
1503
+ "model.language_model.layers.33.self_attn.v_proj": {
1504
+ "quant_algo": "MXFP8",
1505
+ "weight_block_size": [
1506
+ 1,
1507
+ 32
1508
+ ]
1509
+ },
1510
+ "model.language_model.layers.34.mlp.experts": {
1511
+ "group_size": 16,
1512
+ "quant_algo": "NVFP4",
1513
+ "weight_scale_encoding": "csf_32x2_v1"
1514
+ },
1515
+ "model.language_model.layers.34.mlp.shared_experts.down_proj": {
1516
+ "quant_algo": "FP8_PB_WO",
1517
+ "weight_block_size": [
1518
+ 128,
1519
+ 128
1520
+ ]
1521
+ },
1522
+ "model.language_model.layers.34.mlp.shared_experts.gate_proj": {
1523
+ "quant_algo": "FP8_PB_WO",
1524
+ "weight_block_size": [
1525
+ 128,
1526
+ 128
1527
+ ]
1528
+ },
1529
+ "model.language_model.layers.34.mlp.shared_experts.up_proj": {
1530
+ "quant_algo": "FP8_PB_WO",
1531
+ "weight_block_size": [
1532
+ 128,
1533
+ 128
1534
+ ]
1535
+ },
1536
+ "model.language_model.layers.34.self_attn.k_proj": {
1537
+ "quant_algo": "MXFP8",
1538
+ "weight_block_size": [
1539
+ 1,
1540
+ 32
1541
+ ]
1542
+ },
1543
+ "model.language_model.layers.34.self_attn.o_proj": {
1544
+ "quant_algo": "MXFP8",
1545
+ "weight_block_size": [
1546
+ 1,
1547
+ 32
1548
+ ]
1549
+ },
1550
+ "model.language_model.layers.34.self_attn.q_proj": {
1551
+ "quant_algo": "MXFP8",
1552
+ "weight_block_size": [
1553
+ 1,
1554
+ 32
1555
+ ]
1556
+ },
1557
+ "model.language_model.layers.34.self_attn.v_proj": {
1558
+ "quant_algo": "MXFP8",
1559
+ "weight_block_size": [
1560
+ 1,
1561
+ 32
1562
+ ]
1563
+ },
1564
+ "model.language_model.layers.35.mlp.experts": {
1565
+ "group_size": 16,
1566
+ "quant_algo": "NVFP4",
1567
+ "weight_scale_encoding": "csf_32x2_v1"
1568
+ },
1569
+ "model.language_model.layers.35.mlp.shared_experts.down_proj": {
1570
+ "quant_algo": "FP8_PB_WO",
1571
+ "weight_block_size": [
1572
+ 128,
1573
+ 128
1574
+ ]
1575
+ },
1576
+ "model.language_model.layers.35.mlp.shared_experts.gate_proj": {
1577
+ "quant_algo": "FP8_PB_WO",
1578
+ "weight_block_size": [
1579
+ 128,
1580
+ 128
1581
+ ]
1582
+ },
1583
+ "model.language_model.layers.35.mlp.shared_experts.up_proj": {
1584
+ "quant_algo": "FP8_PB_WO",
1585
+ "weight_block_size": [
1586
+ 128,
1587
+ 128
1588
+ ]
1589
+ },
1590
+ "model.language_model.layers.35.self_attn.kv_a_proj_with_mqa": {
1591
+ "quant_algo": "FP8_PB_WO",
1592
+ "weight_block_size": [
1593
+ 128,
1594
+ 128
1595
+ ]
1596
+ },
1597
+ "model.language_model.layers.35.self_attn.o_proj": {
1598
+ "quant_algo": "FP8_PB_WO",
1599
+ "weight_block_size": [
1600
+ 128,
1601
+ 128
1602
+ ]
1603
+ },
1604
+ "model.language_model.layers.35.self_attn.q_a_proj": {
1605
+ "quant_algo": "FP8_PB_WO",
1606
+ "weight_block_size": [
1607
+ 128,
1608
+ 128
1609
+ ]
1610
+ },
1611
+ "model.language_model.layers.35.self_attn.q_b_proj": {
1612
+ "quant_algo": "FP8_PB_WO",
1613
+ "weight_block_size": [
1614
+ 128,
1615
+ 128
1616
+ ]
1617
+ },
1618
+ "model.language_model.layers.36.mlp.experts": {
1619
+ "group_size": 16,
1620
+ "quant_algo": "NVFP4",
1621
+ "weight_scale_encoding": "csf_32x2_v1"
1622
+ },
1623
+ "model.language_model.layers.36.mlp.shared_experts.down_proj": {
1624
+ "quant_algo": "FP8_PB_WO",
1625
+ "weight_block_size": [
1626
+ 128,
1627
+ 128
1628
+ ]
1629
+ },
1630
+ "model.language_model.layers.36.mlp.shared_experts.gate_proj": {
1631
+ "quant_algo": "FP8_PB_WO",
1632
+ "weight_block_size": [
1633
+ 128,
1634
+ 128
1635
+ ]
1636
+ },
1637
+ "model.language_model.layers.36.mlp.shared_experts.up_proj": {
1638
+ "quant_algo": "FP8_PB_WO",
1639
+ "weight_block_size": [
1640
+ 128,
1641
+ 128
1642
+ ]
1643
+ },
1644
+ "model.language_model.layers.36.self_attn.k_proj": {
1645
+ "quant_algo": "MXFP8",
1646
+ "weight_block_size": [
1647
+ 1,
1648
+ 32
1649
+ ]
1650
+ },
1651
+ "model.language_model.layers.36.self_attn.o_proj": {
1652
+ "quant_algo": "MXFP8",
1653
+ "weight_block_size": [
1654
+ 1,
1655
+ 32
1656
+ ]
1657
+ },
1658
+ "model.language_model.layers.36.self_attn.q_proj": {
1659
+ "quant_algo": "MXFP8",
1660
+ "weight_block_size": [
1661
+ 1,
1662
+ 32
1663
+ ]
1664
+ },
1665
+ "model.language_model.layers.36.self_attn.v_proj": {
1666
+ "quant_algo": "MXFP8",
1667
+ "weight_block_size": [
1668
+ 1,
1669
+ 32
1670
+ ]
1671
+ },
1672
+ "model.language_model.layers.37.mlp.experts": {
1673
+ "group_size": 16,
1674
+ "quant_algo": "NVFP4",
1675
+ "weight_scale_encoding": "csf_32x2_v1"
1676
+ },
1677
+ "model.language_model.layers.37.mlp.shared_experts.down_proj": {
1678
+ "quant_algo": "FP8_PB_WO",
1679
+ "weight_block_size": [
1680
+ 128,
1681
+ 128
1682
+ ]
1683
+ },
1684
+ "model.language_model.layers.37.mlp.shared_experts.gate_proj": {
1685
+ "quant_algo": "FP8_PB_WO",
1686
+ "weight_block_size": [
1687
+ 128,
1688
+ 128
1689
+ ]
1690
+ },
1691
+ "model.language_model.layers.37.mlp.shared_experts.up_proj": {
1692
+ "quant_algo": "FP8_PB_WO",
1693
+ "weight_block_size": [
1694
+ 128,
1695
+ 128
1696
+ ]
1697
+ },
1698
+ "model.language_model.layers.37.self_attn.k_proj": {
1699
+ "quant_algo": "MXFP8",
1700
+ "weight_block_size": [
1701
+ 1,
1702
+ 32
1703
+ ]
1704
+ },
1705
+ "model.language_model.layers.37.self_attn.o_proj": {
1706
+ "quant_algo": "MXFP8",
1707
+ "weight_block_size": [
1708
+ 1,
1709
+ 32
1710
+ ]
1711
+ },
1712
+ "model.language_model.layers.37.self_attn.q_proj": {
1713
+ "quant_algo": "MXFP8",
1714
+ "weight_block_size": [
1715
+ 1,
1716
+ 32
1717
+ ]
1718
+ },
1719
+ "model.language_model.layers.37.self_attn.v_proj": {
1720
+ "quant_algo": "MXFP8",
1721
+ "weight_block_size": [
1722
+ 1,
1723
+ 32
1724
+ ]
1725
+ },
1726
+ "model.language_model.layers.38.mlp.experts": {
1727
+ "group_size": 16,
1728
+ "quant_algo": "NVFP4",
1729
+ "weight_scale_encoding": "csf_32x2_v1"
1730
+ },
1731
+ "model.language_model.layers.38.mlp.shared_experts.down_proj": {
1732
+ "quant_algo": "FP8_PB_WO",
1733
+ "weight_block_size": [
1734
+ 128,
1735
+ 128
1736
+ ]
1737
+ },
1738
+ "model.language_model.layers.38.mlp.shared_experts.gate_proj": {
1739
+ "quant_algo": "FP8_PB_WO",
1740
+ "weight_block_size": [
1741
+ 128,
1742
+ 128
1743
+ ]
1744
+ },
1745
+ "model.language_model.layers.38.mlp.shared_experts.up_proj": {
1746
+ "quant_algo": "FP8_PB_WO",
1747
+ "weight_block_size": [
1748
+ 128,
1749
+ 128
1750
+ ]
1751
+ },
1752
+ "model.language_model.layers.38.self_attn.k_proj": {
1753
+ "quant_algo": "MXFP8",
1754
+ "weight_block_size": [
1755
+ 1,
1756
+ 32
1757
+ ]
1758
+ },
1759
+ "model.language_model.layers.38.self_attn.o_proj": {
1760
+ "quant_algo": "MXFP8",
1761
+ "weight_block_size": [
1762
+ 1,
1763
+ 32
1764
+ ]
1765
+ },
1766
+ "model.language_model.layers.38.self_attn.q_proj": {
1767
+ "quant_algo": "MXFP8",
1768
+ "weight_block_size": [
1769
+ 1,
1770
+ 32
1771
+ ]
1772
+ },
1773
+ "model.language_model.layers.38.self_attn.v_proj": {
1774
+ "quant_algo": "MXFP8",
1775
+ "weight_block_size": [
1776
+ 1,
1777
+ 32
1778
+ ]
1779
+ },
1780
+ "model.language_model.layers.39.mlp.experts": {
1781
+ "group_size": 16,
1782
+ "quant_algo": "NVFP4",
1783
+ "weight_scale_encoding": "csf_32x2_v1"
1784
+ },
1785
+ "model.language_model.layers.39.mlp.shared_experts.down_proj": {
1786
+ "quant_algo": "FP8_PB_WO",
1787
+ "weight_block_size": [
1788
+ 128,
1789
+ 128
1790
+ ]
1791
+ },
1792
+ "model.language_model.layers.39.mlp.shared_experts.gate_proj": {
1793
+ "quant_algo": "FP8_PB_WO",
1794
+ "weight_block_size": [
1795
+ 128,
1796
+ 128
1797
+ ]
1798
+ },
1799
+ "model.language_model.layers.39.mlp.shared_experts.up_proj": {
1800
+ "quant_algo": "FP8_PB_WO",
1801
+ "weight_block_size": [
1802
+ 128,
1803
+ 128
1804
+ ]
1805
+ },
1806
+ "model.language_model.layers.39.self_attn.kv_a_proj_with_mqa": {
1807
+ "quant_algo": "FP8_PB_WO",
1808
+ "weight_block_size": [
1809
+ 128,
1810
+ 128
1811
+ ]
1812
+ },
1813
+ "model.language_model.layers.39.self_attn.o_proj": {
1814
+ "quant_algo": "FP8_PB_WO",
1815
+ "weight_block_size": [
1816
+ 128,
1817
+ 128
1818
+ ]
1819
+ },
1820
+ "model.language_model.layers.39.self_attn.q_a_proj": {
1821
+ "quant_algo": "FP8_PB_WO",
1822
+ "weight_block_size": [
1823
+ 128,
1824
+ 128
1825
+ ]
1826
+ },
1827
+ "model.language_model.layers.39.self_attn.q_b_proj": {
1828
+ "quant_algo": "FP8_PB_WO",
1829
+ "weight_block_size": [
1830
+ 128,
1831
+ 128
1832
+ ]
1833
+ },
1834
+ "model.language_model.layers.4.mlp.experts": {
1835
+ "group_size": 16,
1836
+ "quant_algo": "NVFP4",
1837
+ "weight_scale_encoding": "csf_32x2_v1"
1838
+ },
1839
+ "model.language_model.layers.4.mlp.shared_experts.down_proj": {
1840
+ "quant_algo": "FP8_PB_WO",
1841
+ "weight_block_size": [
1842
+ 128,
1843
+ 128
1844
+ ]
1845
+ },
1846
+ "model.language_model.layers.4.mlp.shared_experts.gate_proj": {
1847
+ "quant_algo": "FP8_PB_WO",
1848
+ "weight_block_size": [
1849
+ 128,
1850
+ 128
1851
+ ]
1852
+ },
1853
+ "model.language_model.layers.4.mlp.shared_experts.up_proj": {
1854
+ "quant_algo": "FP8_PB_WO",
1855
+ "weight_block_size": [
1856
+ 128,
1857
+ 128
1858
+ ]
1859
+ },
1860
+ "model.language_model.layers.4.self_attn.k_proj": {
1861
+ "quant_algo": "MXFP8",
1862
+ "weight_block_size": [
1863
+ 1,
1864
+ 32
1865
+ ]
1866
+ },
1867
+ "model.language_model.layers.4.self_attn.o_proj": {
1868
+ "quant_algo": "MXFP8",
1869
+ "weight_block_size": [
1870
+ 1,
1871
+ 32
1872
+ ]
1873
+ },
1874
+ "model.language_model.layers.4.self_attn.q_proj": {
1875
+ "quant_algo": "MXFP8",
1876
+ "weight_block_size": [
1877
+ 1,
1878
+ 32
1879
+ ]
1880
+ },
1881
+ "model.language_model.layers.4.self_attn.v_proj": {
1882
+ "quant_algo": "MXFP8",
1883
+ "weight_block_size": [
1884
+ 1,
1885
+ 32
1886
+ ]
1887
+ },
1888
+ "model.language_model.layers.40.mlp.experts": {
1889
+ "group_size": 16,
1890
+ "quant_algo": "NVFP4",
1891
+ "weight_scale_encoding": "csf_32x2_v1"
1892
+ },
1893
+ "model.language_model.layers.40.mlp.shared_experts.down_proj": {
1894
+ "quant_algo": "FP8_PB_WO",
1895
+ "weight_block_size": [
1896
+ 128,
1897
+ 128
1898
+ ]
1899
+ },
1900
+ "model.language_model.layers.40.mlp.shared_experts.gate_proj": {
1901
+ "quant_algo": "FP8_PB_WO",
1902
+ "weight_block_size": [
1903
+ 128,
1904
+ 128
1905
+ ]
1906
+ },
1907
+ "model.language_model.layers.40.mlp.shared_experts.up_proj": {
1908
+ "quant_algo": "FP8_PB_WO",
1909
+ "weight_block_size": [
1910
+ 128,
1911
+ 128
1912
+ ]
1913
+ },
1914
+ "model.language_model.layers.40.self_attn.k_proj": {
1915
+ "quant_algo": "MXFP8",
1916
+ "weight_block_size": [
1917
+ 1,
1918
+ 32
1919
+ ]
1920
+ },
1921
+ "model.language_model.layers.40.self_attn.o_proj": {
1922
+ "quant_algo": "MXFP8",
1923
+ "weight_block_size": [
1924
+ 1,
1925
+ 32
1926
+ ]
1927
+ },
1928
+ "model.language_model.layers.40.self_attn.q_proj": {
1929
+ "quant_algo": "MXFP8",
1930
+ "weight_block_size": [
1931
+ 1,
1932
+ 32
1933
+ ]
1934
+ },
1935
+ "model.language_model.layers.40.self_attn.v_proj": {
1936
+ "quant_algo": "MXFP8",
1937
+ "weight_block_size": [
1938
+ 1,
1939
+ 32
1940
+ ]
1941
+ },
1942
+ "model.language_model.layers.41.mlp.experts": {
1943
+ "group_size": 16,
1944
+ "quant_algo": "NVFP4",
1945
+ "weight_scale_encoding": "csf_32x2_v1"
1946
+ },
1947
+ "model.language_model.layers.41.mlp.shared_experts.down_proj": {
1948
+ "quant_algo": "FP8_PB_WO",
1949
+ "weight_block_size": [
1950
+ 128,
1951
+ 128
1952
+ ]
1953
+ },
1954
+ "model.language_model.layers.41.mlp.shared_experts.gate_proj": {
1955
+ "quant_algo": "FP8_PB_WO",
1956
+ "weight_block_size": [
1957
+ 128,
1958
+ 128
1959
+ ]
1960
+ },
1961
+ "model.language_model.layers.41.mlp.shared_experts.up_proj": {
1962
+ "quant_algo": "FP8_PB_WO",
1963
+ "weight_block_size": [
1964
+ 128,
1965
+ 128
1966
+ ]
1967
+ },
1968
+ "model.language_model.layers.41.self_attn.k_proj": {
1969
+ "quant_algo": "MXFP8",
1970
+ "weight_block_size": [
1971
+ 1,
1972
+ 32
1973
+ ]
1974
+ },
1975
+ "model.language_model.layers.41.self_attn.o_proj": {
1976
+ "quant_algo": "MXFP8",
1977
+ "weight_block_size": [
1978
+ 1,
1979
+ 32
1980
+ ]
1981
+ },
1982
+ "model.language_model.layers.41.self_attn.q_proj": {
1983
+ "quant_algo": "MXFP8",
1984
+ "weight_block_size": [
1985
+ 1,
1986
+ 32
1987
+ ]
1988
+ },
1989
+ "model.language_model.layers.41.self_attn.v_proj": {
1990
+ "quant_algo": "MXFP8",
1991
+ "weight_block_size": [
1992
+ 1,
1993
+ 32
1994
+ ]
1995
+ },
1996
+ "model.language_model.layers.42.mlp.experts": {
1997
+ "group_size": 16,
1998
+ "quant_algo": "NVFP4",
1999
+ "weight_scale_encoding": "csf_32x2_v1"
2000
+ },
2001
+ "model.language_model.layers.42.mlp.shared_experts.down_proj": {
2002
+ "quant_algo": "FP8_PB_WO",
2003
+ "weight_block_size": [
2004
+ 128,
2005
+ 128
2006
+ ]
2007
+ },
2008
+ "model.language_model.layers.42.mlp.shared_experts.gate_proj": {
2009
+ "quant_algo": "FP8_PB_WO",
2010
+ "weight_block_size": [
2011
+ 128,
2012
+ 128
2013
+ ]
2014
+ },
2015
+ "model.language_model.layers.42.mlp.shared_experts.up_proj": {
2016
+ "quant_algo": "FP8_PB_WO",
2017
+ "weight_block_size": [
2018
+ 128,
2019
+ 128
2020
+ ]
2021
+ },
2022
+ "model.language_model.layers.42.self_attn.k_proj": {
2023
+ "quant_algo": "MXFP8",
2024
+ "weight_block_size": [
2025
+ 1,
2026
+ 32
2027
+ ]
2028
+ },
2029
+ "model.language_model.layers.42.self_attn.o_proj": {
2030
+ "quant_algo": "MXFP8",
2031
+ "weight_block_size": [
2032
+ 1,
2033
+ 32
2034
+ ]
2035
+ },
2036
+ "model.language_model.layers.42.self_attn.q_proj": {
2037
+ "quant_algo": "MXFP8",
2038
+ "weight_block_size": [
2039
+ 1,
2040
+ 32
2041
+ ]
2042
+ },
2043
+ "model.language_model.layers.42.self_attn.v_proj": {
2044
+ "quant_algo": "MXFP8",
2045
+ "weight_block_size": [
2046
+ 1,
2047
+ 32
2048
+ ]
2049
+ },
2050
+ "model.language_model.layers.43.mlp.experts": {
2051
+ "group_size": 16,
2052
+ "quant_algo": "NVFP4",
2053
+ "weight_scale_encoding": "csf_32x2_v1"
2054
+ },
2055
+ "model.language_model.layers.43.mlp.shared_experts.down_proj": {
2056
+ "quant_algo": "FP8_PB_WO",
2057
+ "weight_block_size": [
2058
+ 128,
2059
+ 128
2060
+ ]
2061
+ },
2062
+ "model.language_model.layers.43.mlp.shared_experts.gate_proj": {
2063
+ "quant_algo": "FP8_PB_WO",
2064
+ "weight_block_size": [
2065
+ 128,
2066
+ 128
2067
+ ]
2068
+ },
2069
+ "model.language_model.layers.43.mlp.shared_experts.up_proj": {
2070
+ "quant_algo": "FP8_PB_WO",
2071
+ "weight_block_size": [
2072
+ 128,
2073
+ 128
2074
+ ]
2075
+ },
2076
+ "model.language_model.layers.43.self_attn.kv_a_proj_with_mqa": {
2077
+ "quant_algo": "FP8_PB_WO",
2078
+ "weight_block_size": [
2079
+ 128,
2080
+ 128
2081
+ ]
2082
+ },
2083
+ "model.language_model.layers.43.self_attn.o_proj": {
2084
+ "quant_algo": "FP8_PB_WO",
2085
+ "weight_block_size": [
2086
+ 128,
2087
+ 128
2088
+ ]
2089
+ },
2090
+ "model.language_model.layers.43.self_attn.q_a_proj": {
2091
+ "quant_algo": "FP8_PB_WO",
2092
+ "weight_block_size": [
2093
+ 128,
2094
+ 128
2095
+ ]
2096
+ },
2097
+ "model.language_model.layers.43.self_attn.q_b_proj": {
2098
+ "quant_algo": "FP8_PB_WO",
2099
+ "weight_block_size": [
2100
+ 128,
2101
+ 128
2102
+ ]
2103
+ },
2104
+ "model.language_model.layers.44.mlp.experts": {
2105
+ "group_size": 16,
2106
+ "quant_algo": "NVFP4",
2107
+ "weight_scale_encoding": "csf_32x2_v1"
2108
+ },
2109
+ "model.language_model.layers.44.mlp.shared_experts.down_proj": {
2110
+ "quant_algo": "FP8_PB_WO",
2111
+ "weight_block_size": [
2112
+ 128,
2113
+ 128
2114
+ ]
2115
+ },
2116
+ "model.language_model.layers.44.mlp.shared_experts.gate_proj": {
2117
+ "quant_algo": "FP8_PB_WO",
2118
+ "weight_block_size": [
2119
+ 128,
2120
+ 128
2121
+ ]
2122
+ },
2123
+ "model.language_model.layers.44.mlp.shared_experts.up_proj": {
2124
+ "quant_algo": "FP8_PB_WO",
2125
+ "weight_block_size": [
2126
+ 128,
2127
+ 128
2128
+ ]
2129
+ },
2130
+ "model.language_model.layers.44.self_attn.k_proj": {
2131
+ "quant_algo": "MXFP8",
2132
+ "weight_block_size": [
2133
+ 1,
2134
+ 32
2135
+ ]
2136
+ },
2137
+ "model.language_model.layers.44.self_attn.o_proj": {
2138
+ "quant_algo": "MXFP8",
2139
+ "weight_block_size": [
2140
+ 1,
2141
+ 32
2142
+ ]
2143
+ },
2144
+ "model.language_model.layers.44.self_attn.q_proj": {
2145
+ "quant_algo": "MXFP8",
2146
+ "weight_block_size": [
2147
+ 1,
2148
+ 32
2149
+ ]
2150
+ },
2151
+ "model.language_model.layers.44.self_attn.v_proj": {
2152
+ "quant_algo": "MXFP8",
2153
+ "weight_block_size": [
2154
+ 1,
2155
+ 32
2156
+ ]
2157
+ },
2158
+ "model.language_model.layers.45.mlp.experts": {
2159
+ "group_size": 16,
2160
+ "quant_algo": "W4A16_NVFP4"
2161
+ },
2162
+ "model.language_model.layers.45.mlp.shared_experts.down_proj": {
2163
+ "group_size": 32,
2164
+ "quant_algo": "MXFP8"
2165
+ },
2166
+ "model.language_model.layers.45.mlp.shared_experts.gate_proj": {
2167
+ "group_size": 32,
2168
+ "quant_algo": "MXFP8"
2169
+ },
2170
+ "model.language_model.layers.45.mlp.shared_experts.up_proj": {
2171
+ "group_size": 32,
2172
+ "quant_algo": "MXFP8"
2173
+ },
2174
+ "model.language_model.layers.45.self_attn.indexer.weights_proj": {
2175
+ "group_size": 32,
2176
+ "quant_algo": "MXFP8"
2177
+ },
2178
+ "model.language_model.layers.45.self_attn.indexer.wk": {
2179
+ "group_size": 32,
2180
+ "quant_algo": "MXFP8"
2181
+ },
2182
+ "model.language_model.layers.45.self_attn.indexer.wq_b": {
2183
+ "group_size": 32,
2184
+ "quant_algo": "MXFP8"
2185
+ },
2186
+ "model.language_model.layers.45.self_attn.kv_a_proj_with_mqa": {
2187
+ "group_size": 32,
2188
+ "quant_algo": "MXFP8"
2189
+ },
2190
+ "model.language_model.layers.45.self_attn.kv_b_proj": {
2191
+ "group_size": 32,
2192
+ "quant_algo": "MXFP8"
2193
+ },
2194
+ "model.language_model.layers.45.self_attn.o_proj": {
2195
+ "group_size": 32,
2196
+ "quant_algo": "MXFP8"
2197
+ },
2198
+ "model.language_model.layers.45.self_attn.q_a_proj": {
2199
+ "group_size": 32,
2200
+ "quant_algo": "MXFP8"
2201
+ },
2202
+ "model.language_model.layers.45.self_attn.q_b_proj": {
2203
+ "group_size": 32,
2204
+ "quant_algo": "MXFP8"
2205
+ },
2206
+ "model.language_model.layers.5.mlp.experts": {
2207
+ "group_size": 16,
2208
+ "quant_algo": "NVFP4",
2209
+ "weight_scale_encoding": "csf_32x2_v1"
2210
+ },
2211
+ "model.language_model.layers.5.mlp.shared_experts.down_proj": {
2212
+ "quant_algo": "FP8_PB_WO",
2213
+ "weight_block_size": [
2214
+ 128,
2215
+ 128
2216
+ ]
2217
+ },
2218
+ "model.language_model.layers.5.mlp.shared_experts.gate_proj": {
2219
+ "quant_algo": "FP8_PB_WO",
2220
+ "weight_block_size": [
2221
+ 128,
2222
+ 128
2223
+ ]
2224
+ },
2225
+ "model.language_model.layers.5.mlp.shared_experts.up_proj": {
2226
+ "quant_algo": "FP8_PB_WO",
2227
+ "weight_block_size": [
2228
+ 128,
2229
+ 128
2230
+ ]
2231
+ },
2232
+ "model.language_model.layers.5.self_attn.k_proj": {
2233
+ "quant_algo": "MXFP8",
2234
+ "weight_block_size": [
2235
+ 1,
2236
+ 32
2237
+ ]
2238
+ },
2239
+ "model.language_model.layers.5.self_attn.o_proj": {
2240
+ "quant_algo": "MXFP8",
2241
+ "weight_block_size": [
2242
+ 1,
2243
+ 32
2244
+ ]
2245
+ },
2246
+ "model.language_model.layers.5.self_attn.q_proj": {
2247
+ "quant_algo": "MXFP8",
2248
+ "weight_block_size": [
2249
+ 1,
2250
+ 32
2251
+ ]
2252
+ },
2253
+ "model.language_model.layers.5.self_attn.v_proj": {
2254
+ "quant_algo": "MXFP8",
2255
+ "weight_block_size": [
2256
+ 1,
2257
+ 32
2258
+ ]
2259
+ },
2260
+ "model.language_model.layers.6.mlp.experts": {
2261
+ "group_size": 16,
2262
+ "quant_algo": "NVFP4",
2263
+ "weight_scale_encoding": "csf_32x2_v1"
2264
+ },
2265
+ "model.language_model.layers.6.mlp.shared_experts.down_proj": {
2266
+ "quant_algo": "FP8_PB_WO",
2267
+ "weight_block_size": [
2268
+ 128,
2269
+ 128
2270
+ ]
2271
+ },
2272
+ "model.language_model.layers.6.mlp.shared_experts.gate_proj": {
2273
+ "quant_algo": "FP8_PB_WO",
2274
+ "weight_block_size": [
2275
+ 128,
2276
+ 128
2277
+ ]
2278
+ },
2279
+ "model.language_model.layers.6.mlp.shared_experts.up_proj": {
2280
+ "quant_algo": "FP8_PB_WO",
2281
+ "weight_block_size": [
2282
+ 128,
2283
+ 128
2284
+ ]
2285
+ },
2286
+ "model.language_model.layers.6.self_attn.k_proj": {
2287
+ "quant_algo": "MXFP8",
2288
+ "weight_block_size": [
2289
+ 1,
2290
+ 32
2291
+ ]
2292
+ },
2293
+ "model.language_model.layers.6.self_attn.o_proj": {
2294
+ "quant_algo": "MXFP8",
2295
+ "weight_block_size": [
2296
+ 1,
2297
+ 32
2298
+ ]
2299
+ },
2300
+ "model.language_model.layers.6.self_attn.q_proj": {
2301
+ "quant_algo": "MXFP8",
2302
+ "weight_block_size": [
2303
+ 1,
2304
+ 32
2305
+ ]
2306
+ },
2307
+ "model.language_model.layers.6.self_attn.v_proj": {
2308
+ "quant_algo": "MXFP8",
2309
+ "weight_block_size": [
2310
+ 1,
2311
+ 32
2312
+ ]
2313
+ },
2314
+ "model.language_model.layers.7.mlp.experts": {
2315
+ "group_size": 16,
2316
+ "quant_algo": "NVFP4",
2317
+ "weight_scale_encoding": "csf_32x2_v1"
2318
+ },
2319
+ "model.language_model.layers.7.mlp.shared_experts.down_proj": {
2320
+ "quant_algo": "FP8_PB_WO",
2321
+ "weight_block_size": [
2322
+ 128,
2323
+ 128
2324
+ ]
2325
+ },
2326
+ "model.language_model.layers.7.mlp.shared_experts.gate_proj": {
2327
+ "quant_algo": "FP8_PB_WO",
2328
+ "weight_block_size": [
2329
+ 128,
2330
+ 128
2331
+ ]
2332
+ },
2333
+ "model.language_model.layers.7.mlp.shared_experts.up_proj": {
2334
+ "quant_algo": "FP8_PB_WO",
2335
+ "weight_block_size": [
2336
+ 128,
2337
+ 128
2338
+ ]
2339
+ },
2340
+ "model.language_model.layers.7.self_attn.kv_a_proj_with_mqa": {
2341
+ "quant_algo": "FP8_PB_WO",
2342
+ "weight_block_size": [
2343
+ 128,
2344
+ 128
2345
+ ]
2346
+ },
2347
+ "model.language_model.layers.7.self_attn.o_proj": {
2348
+ "quant_algo": "FP8_PB_WO",
2349
+ "weight_block_size": [
2350
+ 128,
2351
+ 128
2352
+ ]
2353
+ },
2354
+ "model.language_model.layers.7.self_attn.q_a_proj": {
2355
+ "quant_algo": "FP8_PB_WO",
2356
+ "weight_block_size": [
2357
+ 128,
2358
+ 128
2359
+ ]
2360
+ },
2361
+ "model.language_model.layers.7.self_attn.q_b_proj": {
2362
+ "quant_algo": "FP8_PB_WO",
2363
+ "weight_block_size": [
2364
+ 128,
2365
+ 128
2366
+ ]
2367
+ },
2368
+ "model.language_model.layers.8.mlp.experts": {
2369
+ "group_size": 16,
2370
+ "quant_algo": "NVFP4",
2371
+ "weight_scale_encoding": "csf_32x2_v1"
2372
+ },
2373
+ "model.language_model.layers.8.mlp.shared_experts.down_proj": {
2374
+ "quant_algo": "FP8_PB_WO",
2375
+ "weight_block_size": [
2376
+ 128,
2377
+ 128
2378
+ ]
2379
+ },
2380
+ "model.language_model.layers.8.mlp.shared_experts.gate_proj": {
2381
+ "quant_algo": "FP8_PB_WO",
2382
+ "weight_block_size": [
2383
+ 128,
2384
+ 128
2385
+ ]
2386
+ },
2387
+ "model.language_model.layers.8.mlp.shared_experts.up_proj": {
2388
+ "quant_algo": "FP8_PB_WO",
2389
+ "weight_block_size": [
2390
+ 128,
2391
+ 128
2392
+ ]
2393
+ },
2394
+ "model.language_model.layers.8.self_attn.k_proj": {
2395
+ "quant_algo": "MXFP8",
2396
+ "weight_block_size": [
2397
+ 1,
2398
+ 32
2399
+ ]
2400
+ },
2401
+ "model.language_model.layers.8.self_attn.o_proj": {
2402
+ "quant_algo": "MXFP8",
2403
+ "weight_block_size": [
2404
+ 1,
2405
+ 32
2406
+ ]
2407
+ },
2408
+ "model.language_model.layers.8.self_attn.q_proj": {
2409
+ "quant_algo": "MXFP8",
2410
+ "weight_block_size": [
2411
+ 1,
2412
+ 32
2413
+ ]
2414
+ },
2415
+ "model.language_model.layers.8.self_attn.v_proj": {
2416
+ "quant_algo": "MXFP8",
2417
+ "weight_block_size": [
2418
+ 1,
2419
+ 32
2420
+ ]
2421
+ },
2422
+ "model.language_model.layers.9.mlp.experts": {
2423
+ "group_size": 16,
2424
+ "quant_algo": "NVFP4",
2425
+ "weight_scale_encoding": "csf_32x2_v1"
2426
+ },
2427
+ "model.language_model.layers.9.mlp.shared_experts.down_proj": {
2428
+ "quant_algo": "FP8_PB_WO",
2429
+ "weight_block_size": [
2430
+ 128,
2431
+ 128
2432
+ ]
2433
+ },
2434
+ "model.language_model.layers.9.mlp.shared_experts.gate_proj": {
2435
+ "quant_algo": "FP8_PB_WO",
2436
+ "weight_block_size": [
2437
+ 128,
2438
+ 128
2439
+ ]
2440
+ },
2441
+ "model.language_model.layers.9.mlp.shared_experts.up_proj": {
2442
+ "quant_algo": "FP8_PB_WO",
2443
+ "weight_block_size": [
2444
+ 128,
2445
+ 128
2446
+ ]
2447
+ },
2448
+ "model.language_model.layers.9.self_attn.k_proj": {
2449
+ "quant_algo": "MXFP8",
2450
+ "weight_block_size": [
2451
+ 1,
2452
+ 32
2453
+ ]
2454
+ },
2455
+ "model.language_model.layers.9.self_attn.o_proj": {
2456
+ "quant_algo": "MXFP8",
2457
+ "weight_block_size": [
2458
+ 1,
2459
+ 32
2460
+ ]
2461
+ },
2462
+ "model.language_model.layers.9.self_attn.q_proj": {
2463
+ "quant_algo": "MXFP8",
2464
+ "weight_block_size": [
2465
+ 1,
2466
+ 32
2467
+ ]
2468
+ },
2469
+ "model.language_model.layers.9.self_attn.v_proj": {
2470
+ "quant_algo": "MXFP8",
2471
+ "weight_block_size": [
2472
+ 1,
2473
+ 32
2474
+ ]
2475
+ },
2476
+ "model.visual.blocks.0.attn.proj": {
2477
+ "group_size": 32,
2478
+ "quant_algo": "MXFP8"
2479
+ },
2480
+ "model.visual.blocks.0.attn.qkv": {
2481
+ "group_size": 32,
2482
+ "quant_algo": "MXFP8"
2483
+ },
2484
+ "model.visual.blocks.0.mlp.down_proj": {
2485
+ "group_size": 16,
2486
+ "quant_algo": "W4A16_NVFP4"
2487
+ },
2488
+ "model.visual.blocks.0.mlp.gate_proj": {
2489
+ "group_size": 16,
2490
+ "quant_algo": "W4A16_NVFP4"
2491
+ },
2492
+ "model.visual.blocks.0.mlp.up_proj": {
2493
+ "group_size": 16,
2494
+ "quant_algo": "W4A16_NVFP4"
2495
+ },
2496
+ "model.visual.blocks.1.attn.proj": {
2497
+ "group_size": 32,
2498
+ "quant_algo": "MXFP8"
2499
+ },
2500
+ "model.visual.blocks.1.attn.qkv": {
2501
+ "group_size": 32,
2502
+ "quant_algo": "MXFP8"
2503
+ },
2504
+ "model.visual.blocks.1.mlp.down_proj": {
2505
+ "group_size": 16,
2506
+ "quant_algo": "W4A16_NVFP4"
2507
+ },
2508
+ "model.visual.blocks.1.mlp.gate_proj": {
2509
+ "group_size": 16,
2510
+ "quant_algo": "W4A16_NVFP4"
2511
+ },
2512
+ "model.visual.blocks.1.mlp.up_proj": {
2513
+ "group_size": 16,
2514
+ "quant_algo": "W4A16_NVFP4"
2515
+ },
2516
+ "model.visual.blocks.10.attn.proj": {
2517
+ "group_size": 32,
2518
+ "quant_algo": "MXFP8"
2519
+ },
2520
+ "model.visual.blocks.10.attn.qkv": {
2521
+ "group_size": 32,
2522
+ "quant_algo": "MXFP8"
2523
+ },
2524
+ "model.visual.blocks.10.mlp.down_proj": {
2525
+ "group_size": 16,
2526
+ "quant_algo": "W4A16_NVFP4"
2527
+ },
2528
+ "model.visual.blocks.10.mlp.gate_proj": {
2529
+ "group_size": 16,
2530
+ "quant_algo": "W4A16_NVFP4"
2531
+ },
2532
+ "model.visual.blocks.10.mlp.up_proj": {
2533
+ "group_size": 16,
2534
+ "quant_algo": "W4A16_NVFP4"
2535
+ },
2536
+ "model.visual.blocks.11.attn.proj": {
2537
+ "group_size": 32,
2538
+ "quant_algo": "MXFP8"
2539
+ },
2540
+ "model.visual.blocks.11.attn.qkv": {
2541
+ "group_size": 32,
2542
+ "quant_algo": "MXFP8"
2543
+ },
2544
+ "model.visual.blocks.11.mlp.down_proj": {
2545
+ "group_size": 16,
2546
+ "quant_algo": "W4A16_NVFP4"
2547
+ },
2548
+ "model.visual.blocks.11.mlp.gate_proj": {
2549
+ "group_size": 16,
2550
+ "quant_algo": "W4A16_NVFP4"
2551
+ },
2552
+ "model.visual.blocks.11.mlp.up_proj": {
2553
+ "group_size": 16,
2554
+ "quant_algo": "W4A16_NVFP4"
2555
+ },
2556
+ "model.visual.blocks.12.attn.proj": {
2557
+ "group_size": 32,
2558
+ "quant_algo": "MXFP8"
2559
+ },
2560
+ "model.visual.blocks.12.attn.qkv": {
2561
+ "group_size": 32,
2562
+ "quant_algo": "MXFP8"
2563
+ },
2564
+ "model.visual.blocks.12.mlp.down_proj": {
2565
+ "group_size": 16,
2566
+ "quant_algo": "W4A16_NVFP4"
2567
+ },
2568
+ "model.visual.blocks.12.mlp.gate_proj": {
2569
+ "group_size": 16,
2570
+ "quant_algo": "W4A16_NVFP4"
2571
+ },
2572
+ "model.visual.blocks.12.mlp.up_proj": {
2573
+ "group_size": 16,
2574
+ "quant_algo": "W4A16_NVFP4"
2575
+ },
2576
+ "model.visual.blocks.13.attn.proj": {
2577
+ "group_size": 32,
2578
+ "quant_algo": "MXFP8"
2579
+ },
2580
+ "model.visual.blocks.13.attn.qkv": {
2581
+ "group_size": 32,
2582
+ "quant_algo": "MXFP8"
2583
+ },
2584
+ "model.visual.blocks.13.mlp.down_proj": {
2585
+ "group_size": 16,
2586
+ "quant_algo": "W4A16_NVFP4"
2587
+ },
2588
+ "model.visual.blocks.13.mlp.gate_proj": {
2589
+ "group_size": 16,
2590
+ "quant_algo": "W4A16_NVFP4"
2591
+ },
2592
+ "model.visual.blocks.13.mlp.up_proj": {
2593
+ "group_size": 16,
2594
+ "quant_algo": "W4A16_NVFP4"
2595
+ },
2596
+ "model.visual.blocks.14.attn.proj": {
2597
+ "group_size": 32,
2598
+ "quant_algo": "MXFP8"
2599
+ },
2600
+ "model.visual.blocks.14.attn.qkv": {
2601
+ "group_size": 32,
2602
+ "quant_algo": "MXFP8"
2603
+ },
2604
+ "model.visual.blocks.14.mlp.down_proj": {
2605
+ "group_size": 16,
2606
+ "quant_algo": "W4A16_NVFP4"
2607
+ },
2608
+ "model.visual.blocks.14.mlp.gate_proj": {
2609
+ "group_size": 16,
2610
+ "quant_algo": "W4A16_NVFP4"
2611
+ },
2612
+ "model.visual.blocks.14.mlp.up_proj": {
2613
+ "group_size": 16,
2614
+ "quant_algo": "W4A16_NVFP4"
2615
+ },
2616
+ "model.visual.blocks.15.attn.proj": {
2617
+ "group_size": 32,
2618
+ "quant_algo": "MXFP8"
2619
+ },
2620
+ "model.visual.blocks.15.attn.qkv": {
2621
+ "group_size": 32,
2622
+ "quant_algo": "MXFP8"
2623
+ },
2624
+ "model.visual.blocks.15.mlp.down_proj": {
2625
+ "group_size": 16,
2626
+ "quant_algo": "W4A16_NVFP4"
2627
+ },
2628
+ "model.visual.blocks.15.mlp.gate_proj": {
2629
+ "group_size": 16,
2630
+ "quant_algo": "W4A16_NVFP4"
2631
+ },
2632
+ "model.visual.blocks.15.mlp.up_proj": {
2633
+ "group_size": 16,
2634
+ "quant_algo": "W4A16_NVFP4"
2635
+ },
2636
+ "model.visual.blocks.16.attn.proj": {
2637
+ "group_size": 32,
2638
+ "quant_algo": "MXFP8"
2639
+ },
2640
+ "model.visual.blocks.16.attn.qkv": {
2641
+ "group_size": 32,
2642
+ "quant_algo": "MXFP8"
2643
+ },
2644
+ "model.visual.blocks.16.mlp.down_proj": {
2645
+ "group_size": 16,
2646
+ "quant_algo": "W4A16_NVFP4"
2647
+ },
2648
+ "model.visual.blocks.16.mlp.gate_proj": {
2649
+ "group_size": 16,
2650
+ "quant_algo": "W4A16_NVFP4"
2651
+ },
2652
+ "model.visual.blocks.16.mlp.up_proj": {
2653
+ "group_size": 16,
2654
+ "quant_algo": "W4A16_NVFP4"
2655
+ },
2656
+ "model.visual.blocks.17.attn.proj": {
2657
+ "group_size": 32,
2658
+ "quant_algo": "MXFP8"
2659
+ },
2660
+ "model.visual.blocks.17.attn.qkv": {
2661
+ "group_size": 32,
2662
+ "quant_algo": "MXFP8"
2663
+ },
2664
+ "model.visual.blocks.17.mlp.down_proj": {
2665
+ "group_size": 16,
2666
+ "quant_algo": "W4A16_NVFP4"
2667
+ },
2668
+ "model.visual.blocks.17.mlp.gate_proj": {
2669
+ "group_size": 16,
2670
+ "quant_algo": "W4A16_NVFP4"
2671
+ },
2672
+ "model.visual.blocks.17.mlp.up_proj": {
2673
+ "group_size": 16,
2674
+ "quant_algo": "W4A16_NVFP4"
2675
+ },
2676
+ "model.visual.blocks.18.attn.proj": {
2677
+ "group_size": 32,
2678
+ "quant_algo": "MXFP8"
2679
+ },
2680
+ "model.visual.blocks.18.attn.qkv": {
2681
+ "group_size": 32,
2682
+ "quant_algo": "MXFP8"
2683
+ },
2684
+ "model.visual.blocks.18.mlp.down_proj": {
2685
+ "group_size": 16,
2686
+ "quant_algo": "W4A16_NVFP4"
2687
+ },
2688
+ "model.visual.blocks.18.mlp.gate_proj": {
2689
+ "group_size": 16,
2690
+ "quant_algo": "W4A16_NVFP4"
2691
+ },
2692
+ "model.visual.blocks.18.mlp.up_proj": {
2693
+ "group_size": 16,
2694
+ "quant_algo": "W4A16_NVFP4"
2695
+ },
2696
+ "model.visual.blocks.19.attn.proj": {
2697
+ "group_size": 32,
2698
+ "quant_algo": "MXFP8"
2699
+ },
2700
+ "model.visual.blocks.19.attn.qkv": {
2701
+ "group_size": 32,
2702
+ "quant_algo": "MXFP8"
2703
+ },
2704
+ "model.visual.blocks.19.mlp.down_proj": {
2705
+ "group_size": 16,
2706
+ "quant_algo": "W4A16_NVFP4"
2707
+ },
2708
+ "model.visual.blocks.19.mlp.gate_proj": {
2709
+ "group_size": 16,
2710
+ "quant_algo": "W4A16_NVFP4"
2711
+ },
2712
+ "model.visual.blocks.19.mlp.up_proj": {
2713
+ "group_size": 16,
2714
+ "quant_algo": "W4A16_NVFP4"
2715
+ },
2716
+ "model.visual.blocks.2.attn.proj": {
2717
+ "group_size": 32,
2718
+ "quant_algo": "MXFP8"
2719
+ },
2720
+ "model.visual.blocks.2.attn.qkv": {
2721
+ "group_size": 32,
2722
+ "quant_algo": "MXFP8"
2723
+ },
2724
+ "model.visual.blocks.2.mlp.down_proj": {
2725
+ "group_size": 16,
2726
+ "quant_algo": "W4A16_NVFP4"
2727
+ },
2728
+ "model.visual.blocks.2.mlp.gate_proj": {
2729
+ "group_size": 16,
2730
+ "quant_algo": "W4A16_NVFP4"
2731
+ },
2732
+ "model.visual.blocks.2.mlp.up_proj": {
2733
+ "group_size": 16,
2734
+ "quant_algo": "W4A16_NVFP4"
2735
+ },
2736
+ "model.visual.blocks.20.attn.proj": {
2737
+ "group_size": 32,
2738
+ "quant_algo": "MXFP8"
2739
+ },
2740
+ "model.visual.blocks.20.attn.qkv": {
2741
+ "group_size": 32,
2742
+ "quant_algo": "MXFP8"
2743
+ },
2744
+ "model.visual.blocks.20.mlp.down_proj": {
2745
+ "group_size": 16,
2746
+ "quant_algo": "W4A16_NVFP4"
2747
+ },
2748
+ "model.visual.blocks.20.mlp.gate_proj": {
2749
+ "group_size": 16,
2750
+ "quant_algo": "W4A16_NVFP4"
2751
+ },
2752
+ "model.visual.blocks.20.mlp.up_proj": {
2753
+ "group_size": 16,
2754
+ "quant_algo": "W4A16_NVFP4"
2755
+ },
2756
+ "model.visual.blocks.21.attn.proj": {
2757
+ "group_size": 32,
2758
+ "quant_algo": "MXFP8"
2759
+ },
2760
+ "model.visual.blocks.21.attn.qkv": {
2761
+ "group_size": 32,
2762
+ "quant_algo": "MXFP8"
2763
+ },
2764
+ "model.visual.blocks.21.mlp.down_proj": {
2765
+ "group_size": 16,
2766
+ "quant_algo": "W4A16_NVFP4"
2767
+ },
2768
+ "model.visual.blocks.21.mlp.gate_proj": {
2769
+ "group_size": 16,
2770
+ "quant_algo": "W4A16_NVFP4"
2771
+ },
2772
+ "model.visual.blocks.21.mlp.up_proj": {
2773
+ "group_size": 16,
2774
+ "quant_algo": "W4A16_NVFP4"
2775
+ },
2776
+ "model.visual.blocks.22.attn.proj": {
2777
+ "group_size": 32,
2778
+ "quant_algo": "MXFP8"
2779
+ },
2780
+ "model.visual.blocks.22.attn.qkv": {
2781
+ "group_size": 32,
2782
+ "quant_algo": "MXFP8"
2783
+ },
2784
+ "model.visual.blocks.22.mlp.down_proj": {
2785
+ "group_size": 16,
2786
+ "quant_algo": "W4A16_NVFP4"
2787
+ },
2788
+ "model.visual.blocks.22.mlp.gate_proj": {
2789
+ "group_size": 16,
2790
+ "quant_algo": "W4A16_NVFP4"
2791
+ },
2792
+ "model.visual.blocks.22.mlp.up_proj": {
2793
+ "group_size": 16,
2794
+ "quant_algo": "W4A16_NVFP4"
2795
+ },
2796
+ "model.visual.blocks.23.attn.proj": {
2797
+ "group_size": 32,
2798
+ "quant_algo": "MXFP8"
2799
+ },
2800
+ "model.visual.blocks.23.attn.qkv": {
2801
+ "group_size": 32,
2802
+ "quant_algo": "MXFP8"
2803
+ },
2804
+ "model.visual.blocks.23.mlp.down_proj": {
2805
+ "group_size": 16,
2806
+ "quant_algo": "W4A16_NVFP4"
2807
+ },
2808
+ "model.visual.blocks.23.mlp.gate_proj": {
2809
+ "group_size": 16,
2810
+ "quant_algo": "W4A16_NVFP4"
2811
+ },
2812
+ "model.visual.blocks.23.mlp.up_proj": {
2813
+ "group_size": 16,
2814
+ "quant_algo": "W4A16_NVFP4"
2815
+ },
2816
+ "model.visual.blocks.3.attn.proj": {
2817
+ "group_size": 32,
2818
+ "quant_algo": "MXFP8"
2819
+ },
2820
+ "model.visual.blocks.3.attn.qkv": {
2821
+ "group_size": 32,
2822
+ "quant_algo": "MXFP8"
2823
+ },
2824
+ "model.visual.blocks.3.mlp.down_proj": {
2825
+ "group_size": 16,
2826
+ "quant_algo": "W4A16_NVFP4"
2827
+ },
2828
+ "model.visual.blocks.3.mlp.gate_proj": {
2829
+ "group_size": 16,
2830
+ "quant_algo": "W4A16_NVFP4"
2831
+ },
2832
+ "model.visual.blocks.3.mlp.up_proj": {
2833
+ "group_size": 16,
2834
+ "quant_algo": "W4A16_NVFP4"
2835
+ },
2836
+ "model.visual.blocks.4.attn.proj": {
2837
+ "group_size": 32,
2838
+ "quant_algo": "MXFP8"
2839
+ },
2840
+ "model.visual.blocks.4.attn.qkv": {
2841
+ "group_size": 32,
2842
+ "quant_algo": "MXFP8"
2843
+ },
2844
+ "model.visual.blocks.4.mlp.down_proj": {
2845
+ "group_size": 16,
2846
+ "quant_algo": "W4A16_NVFP4"
2847
+ },
2848
+ "model.visual.blocks.4.mlp.gate_proj": {
2849
+ "group_size": 16,
2850
+ "quant_algo": "W4A16_NVFP4"
2851
+ },
2852
+ "model.visual.blocks.4.mlp.up_proj": {
2853
+ "group_size": 16,
2854
+ "quant_algo": "W4A16_NVFP4"
2855
+ },
2856
+ "model.visual.blocks.5.attn.proj": {
2857
+ "group_size": 32,
2858
+ "quant_algo": "MXFP8"
2859
+ },
2860
+ "model.visual.blocks.5.attn.qkv": {
2861
+ "group_size": 32,
2862
+ "quant_algo": "MXFP8"
2863
+ },
2864
+ "model.visual.blocks.5.mlp.down_proj": {
2865
+ "group_size": 16,
2866
+ "quant_algo": "W4A16_NVFP4"
2867
+ },
2868
+ "model.visual.blocks.5.mlp.gate_proj": {
2869
+ "group_size": 16,
2870
+ "quant_algo": "W4A16_NVFP4"
2871
+ },
2872
+ "model.visual.blocks.5.mlp.up_proj": {
2873
+ "group_size": 16,
2874
+ "quant_algo": "W4A16_NVFP4"
2875
+ },
2876
+ "model.visual.blocks.6.attn.proj": {
2877
+ "group_size": 32,
2878
+ "quant_algo": "MXFP8"
2879
+ },
2880
+ "model.visual.blocks.6.attn.qkv": {
2881
+ "group_size": 32,
2882
+ "quant_algo": "MXFP8"
2883
+ },
2884
+ "model.visual.blocks.6.mlp.down_proj": {
2885
+ "group_size": 16,
2886
+ "quant_algo": "W4A16_NVFP4"
2887
+ },
2888
+ "model.visual.blocks.6.mlp.gate_proj": {
2889
+ "group_size": 16,
2890
+ "quant_algo": "W4A16_NVFP4"
2891
+ },
2892
+ "model.visual.blocks.6.mlp.up_proj": {
2893
+ "group_size": 16,
2894
+ "quant_algo": "W4A16_NVFP4"
2895
+ },
2896
+ "model.visual.blocks.7.attn.proj": {
2897
+ "group_size": 32,
2898
+ "quant_algo": "MXFP8"
2899
+ },
2900
+ "model.visual.blocks.7.attn.qkv": {
2901
+ "group_size": 32,
2902
+ "quant_algo": "MXFP8"
2903
+ },
2904
+ "model.visual.blocks.7.mlp.down_proj": {
2905
+ "group_size": 16,
2906
+ "quant_algo": "W4A16_NVFP4"
2907
+ },
2908
+ "model.visual.blocks.7.mlp.gate_proj": {
2909
+ "group_size": 16,
2910
+ "quant_algo": "W4A16_NVFP4"
2911
+ },
2912
+ "model.visual.blocks.7.mlp.up_proj": {
2913
+ "group_size": 16,
2914
+ "quant_algo": "W4A16_NVFP4"
2915
+ },
2916
+ "model.visual.blocks.8.attn.proj": {
2917
+ "group_size": 32,
2918
+ "quant_algo": "MXFP8"
2919
+ },
2920
+ "model.visual.blocks.8.attn.qkv": {
2921
+ "group_size": 32,
2922
+ "quant_algo": "MXFP8"
2923
+ },
2924
+ "model.visual.blocks.8.mlp.down_proj": {
2925
+ "group_size": 16,
2926
+ "quant_algo": "W4A16_NVFP4"
2927
+ },
2928
+ "model.visual.blocks.8.mlp.gate_proj": {
2929
+ "group_size": 16,
2930
+ "quant_algo": "W4A16_NVFP4"
2931
+ },
2932
+ "model.visual.blocks.8.mlp.up_proj": {
2933
+ "group_size": 16,
2934
+ "quant_algo": "W4A16_NVFP4"
2935
+ },
2936
+ "model.visual.blocks.9.attn.proj": {
2937
+ "group_size": 32,
2938
+ "quant_algo": "MXFP8"
2939
+ },
2940
+ "model.visual.blocks.9.attn.qkv": {
2941
+ "group_size": 32,
2942
+ "quant_algo": "MXFP8"
2943
+ },
2944
+ "model.visual.blocks.9.mlp.down_proj": {
2945
+ "group_size": 16,
2946
+ "quant_algo": "W4A16_NVFP4"
2947
+ },
2948
+ "model.visual.blocks.9.mlp.gate_proj": {
2949
+ "group_size": 16,
2950
+ "quant_algo": "W4A16_NVFP4"
2951
+ },
2952
+ "model.visual.blocks.9.mlp.up_proj": {
2953
+ "group_size": 16,
2954
+ "quant_algo": "W4A16_NVFP4"
2955
+ },
2956
+ "model.visual.merger.down_proj": {
2957
+ "group_size": 16,
2958
+ "quant_algo": "W4A16_NVFP4"
2959
+ },
2960
+ "model.visual.merger.gate_proj": {
2961
+ "group_size": 16,
2962
+ "quant_algo": "W4A16_NVFP4"
2963
+ },
2964
+ "model.visual.merger.proj": {
2965
+ "group_size": 16,
2966
+ "quant_algo": "W4A16_NVFP4"
2967
+ },
2968
+ "model.visual.merger.up_proj": {
2969
+ "group_size": 16,
2970
+ "quant_algo": "W4A16_NVFP4"
2971
+ }
2972
+ }
2973
+ }
tensors/model-00001-of-00036.safetensors → model-00001-of-00040.safetensors RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:bee2ce859f246fa647a28a6b8086d2d77c3388e878e135ec851c3c15f1283be4
3
- size 4997009796
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6d2fcce2097ab08926290ff099b792c688a9eb014a6963ba201e6f6c18767730
3
+ size 4294380516
tensors/model-00002-of-00036.safetensors → model-00002-of-00040.safetensors RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:58c17a9a8bb87bc108b2243d0b3554a0fd8e72a668f88bd4c050785a0716ada6
3
- size 4997194608
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:abe5800514c86b4fe57a9f8419a1343460c5459495f626a15c143c077f0eed59
3
+ size 398331164
tensors/model-00003-of-00036.safetensors → model-00003-of-00040.safetensors RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5725f708d20f7089f589b020fa1f83f21d62e20c3c8ed5798aa2385c154b011d
3
- size 4997082968
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a6bf622464604faa5e7d8f3c7c265242fc768f90f5069bc5bff84d5665c3722f
3
+ size 4187354632
tensors/model-00004-of-00036.safetensors → model-00004-of-00040.safetensors RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:15d33f9a63251c4fd63d9c7f9a2477003f1fcbbd5aaab4fc9f47261bf63def12
3
- size 4996991824
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1bf321c5b0c6a866f1916587cb84e0f67b6527fc9da1f432ce8ddb1eeefaf89f
3
+ size 4059621056
model-00005-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fa93cdc2b83135bfd48c38a88e5d9299132e8b0d919ed2e68343fce8c7f1c844
3
+ size 1204588600
model-00006-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7f36a0aebce974e605a66c07c10963671753823640e8d54584a98fd04578414a
3
+ size 3386693718
model-00007-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:08cf34e007f01060b73d63b9e73e3fb7429aef998e61bb521d927a5e1c836ca4
3
+ size 3386630980
model-00008-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:27665caafba3db19034967c9f6c91e9614c0b71270a2a5983131ff845b34c66a
3
+ size 3386784380
model-00009-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6fefef9818a88957a172129075db03351ff8705f912d187439650b548e882d1f
3
+ size 3386714856
model-00010-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:618704500d946c3c573e9eca8ca74f0395139e190a76687feff6930201c32c02
3
+ size 3386642180
model-00011-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e79fd0c2161ffd83b02aba108713a0864959e26cb36cedaca5c1fecd1328d29b
3
+ size 3386705796
model-00012-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cc3ce32db330f4edaed1a814da5709e816b2b5874c582124cb2a319c123f0fb5
3
+ size 3386704188
model-00013-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e44d1253f3fa790150f3c869c3fd04d692c08fcf29f54e9ed0f86f1e3804f44e
3
+ size 3386634084
model-00014-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b9242f5c32541c58f9e9850091664f7359c6d4536254c5f808b68f133487404f
3
+ size 3386785778
model-00015-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:856b041abe136f7bdf624df0e3f816999020e0ceaa44339f8f914cfe6423686c
3
+ size 5080037446
model-00016-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4cc6b1eca1fd60ef05f4ed8b58fc31fc852fde068544fc1e06badcfae51e9dba
3
+ size 5080062512
model-00017-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2dc2ef526eccd2dc1df753aa6c22529a12340270b241f6429d915d43c8ffaf37
3
+ size 5080033556
model-00018-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e1fa1a86341b82892fb5ed234ad82d7836d3fcf004675a000e366116a423144c
3
+ size 5080002106
model-00019-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:49ed0fd14feeb265c8c80e68b007feb6612f220b17cd7bec0382cf4fccdedad3
3
+ size 5080044836
model-00020-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36d13ead5eff2bebc082ea8bfff2cd7952249e3d464e4691f5b33a7334095da8
3
+ size 5080015448
model-00021-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be455e23d8170063b8ac2d22be604e71594fe72755c36b31272253cf8f94447c
3
+ size 5080033720
model-00022-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7f7a0f6aeea5a2ff95b7c2f993675eca757cb3bd7e0ed9876850f73c0c640393
3
+ size 5080007626
model-00023-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:18d7db72e3737476c604f6f23615318158e91026226eafcc3975d23293e36bbe
3
+ size 5080036888
model-00024-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:937b0331965662e4d53e7266006bf9268dcf6e511354486396eb2e1cd1e1fce5
3
+ size 5079998124
model-00025-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:522b327ece804e547f601a266be74cbdd4dfbc2938419052d43a5e1283ba94f5
3
+ size 5080015080
model-00026-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fa5956b9758a382ff3a63a5c4625dd2a876183a8f6083bac1b665a7eb93903e1
3
+ size 5080002452
model-00027-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3e2b7685470b601aceebe226b26b52464fa40d74b9b346af337d45e8faacc1cc
3
+ size 5080025960
model-00028-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:83df1ae7d43eaf70b29fec3194851cb924775b8671440ed02ff2ac066cbaf7e5
3
+ size 5080010694
model-00029-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:223f763f1a1569d664f3e00641043bd6a0a0fe6c7881126620ef0183235fd5cd
3
+ size 5080031820
model-00030-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36e81695f6ddab8108aeb7479b7015a21b8ddb9057b4375527c13fb29af38392
3
+ size 5080122474
model-00031-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:697493ca2ce3f7fc975aaf9d8e65a2b373a2b4331035a4f5379ee4904688c73f
3
+ size 5080041150
model-00032-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1967e72e60ed152b39ab0d951a3c524a885c6c2fb760e55688a08ccf197ae30b
3
+ size 5080082340
model-00033-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3896f4b763903405bfdf37249296bb05f525ca2134541dab7ea91c5513047a1c
3
+ size 5080023370
model-00034-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:50d2cd8d608d3b8c59bd4c654275d6f840cd7598aeb42a2a5da8db75ae5c8b4d
3
+ size 5079989500
model-00035-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:27ba1ee0f211a331fff5221ffe5f00b06eeb002f1865ae8d4fd8335d2c7f4c8c
3
+ size 5080032548
model-00036-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b6954778ce40d39efce3a07c01e1d245e757a701d72ead7a73e48fc2a57cbddd
3
+ size 5080044714
model-00037-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2a956ea3bd9468184565c5e94211c663bd401b3c1c57da08e96d5fd2b49419d9
3
+ size 5080045746
model-00038-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c12da94d3c1642f5ec34d1c1f573d193c75818bfc13b4774954cbd1ec9bfb646
3
+ size 5080051898
model-00039-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5262d01f162e665c5da0f4e841d450e2396b4e45727b704c0ac1e514e287e7a9
3
+ size 5080035462
model-00040-of-00040.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:73342fa305b49ec6677423b090479bbd8d021cd7e9f0afc27c2eeaa0b678dc23
3
+ size 5080022666
model-inputscales.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d3dd6c79b081f1b61bd13a9354a0753f5b0406e3a4e8de05c45ff5c74e56ef1b
3
+ size 4726696
model.safetensors.index.json CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:568c770e4a083ab53c3d74deab348d541724f23ec9637d83513ecb8fa4a7ef73
3
- size 23449767
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1981941b2fabb9271ae3a3cb76be6ea41927f730b6633471d3c147106703aeea
3
+ size 35266806
tensors/model-00005-of-00036.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:8e7089fdbb56f76745f5ed1488d5610c181a77bab476b2c0cd18b83d52b650bf
3
- size 4997092464
 
 
 
 
tensors/model-00006-of-00036.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:5cc20bd94516fadc06153d81466401420cfc345b6bc193c948ce7c4bf020cec1
3
- size 4996967984
 
 
 
 
tensors/model-00007-of-00036.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:ac91e97bc6aa6cf55342bd91cfa03d20d0a61af7b7dcfb7eee157a9e51309372
3
- size 4997293804
 
 
 
 
tensors/model-00008-of-00036.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:3fec2e3f9d1fdac465afb28d77e17bbfa721362aacca61e445c4d3db7fd64c43
3
- size 4996970628
 
 
 
 
tensors/model-00009-of-00036.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:c075d2b39a2ba0e05d826fa35744bc453fa2550a3c8c23437c3b65bbc0c61082
3
- size 4997192808
 
 
 
 
tensors/model-00010-of-00036.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:3e16cb82de5f953a6d88bf58de8f09037eada207faea50b0b4e5b97b4fe26e35
3
- size 4996948996