diff --git a/README.md b/README.md
index 1bfafff765ccb7e1af23d90456f5168465fec7d0..8d0b63efe83b7991a7385aec7791339bfbaa159c 100644
--- a/README.md
+++ b/README.md
@@ -27,16 +27,18 @@ Apple Silicon (MLX) **mixed-precision** quantization of [zai-org/GLM-5.2](https:
 
 **New:** the checkpoint now ships GLM-5.2's **native MTP (nextn) layer** for self-speculative decoding (`--mtp` — see [*Native MTP*](#native-mtp--self-speculative-decoding)). Fully backward-compatible: runtimes without MTP support strip the extra layer and behave exactly as before.
 
-**Update (2026-07): DWQ-retuned weights.** `main` now carries quantization scales/biases re-tuned with **layerwise DWQ** ([alis-dwq](https://github.com/avlp12/alis-dwq); upstream flag: [mlx-lm#1499](https://github.com/ml-explore/mlx-lm/pull/1499)) against the [4.5 bpw sibling](https://huggingface.co/avlp12/GLM-5.2-Alis-MLX-Dynamic-4.5bpw) as teacher, with a 45%-ZH calibration mix — low-bit damage concentrates in the model's Chinese mass, and this targets exactly that. Same recipe, size class and speed; distributional fidelity jumps:
+**Update 2 (2026-07): anchor-guarded clip + DWQ re-run.** `main` is now rebuilt from a **Q8 source** with an anchor-guarded clip search (per-group outlier clipping at max-err slack 1.1 that preserves the min-max anchor weights; method in [alis-dwq](https://github.com/avlp12/alis-dwq)), then DWQ-retuned with the same recipe as before (4.5 bpw-teacher targets, 45%-ZH calibration, upstream flag [mlx-lm#1499](https://github.com/ml-explore/mlx-lm/pull/1499)). Held-out perplexity improves or holds everywhere vs the previous DWQ main: strided wikitext **3.774 → 3.698 (−2.0%)**, code **2.069 → 2.054**, tulu-3 flat (3.564 → 3.571 ±0.027); the clip alone moves the raw pre-DWQ build 4.34 → 4.17. The checkpoint also sheds 4.5 GB: the MTP layer used to ship twice (a module-named copy inside the base shards plus the standalone MTP shard), and now ships once.
 
-| KL / top-1 flip vs the 4.5 bpw reference (T=3072) | pre-DWQ | **DWQ (main)** |
-|---|---|---|
-| EN | 0.727 / 24.9% | **0.383 / 15.6%** |
-| code | 0.252 / 12.7% | **0.193 / 10.2%** |
-| ZH | 0.987 / 35.7% | **0.562 / 21.9%** |
-| overall | 0.655 / 24.4% | **0.379 / 15.9% (−42% KL)** |
+| KL / top-1 flip vs the 4.5 bpw reference (T=3072) | pre-DWQ | first DWQ (`dwq-noclip`) | **clip+DWQ (main)** |
+|---|---|---|---|
+| EN | 0.727 / 24.9% | 0.383 / 15.6% | 0.464 / 17.3% |
+| code | 0.252 / 12.7% | 0.193 / 10.2% | **0.183 / 10.0%** |
+| ZH | 0.987 / 35.7% | 0.562 / 21.9% | 0.610 / 24.8% |
+| overall | 0.655 / 24.4% | 0.379 / 15.9% | 0.419 / 17.4% |
+
+*How to read this table: the reference is the 4.5 bpw **quantized** sibling, not ground truth. The first DWQ was derived from the same 4-bit-lattice family as that reference, so it sits unusually close to it; the clip step re-derives every tensor from a Q8 source (removing the lattice-resonance failure mode), which drifts slightly away from this particular reference even as held-out perplexity — the metric that tracks real quality — improves. Judge builds by the PPL numbers.* Greedy loop-probe (256 tok × EN/code/ZH): clean, no cycles.
 
-The pre-DWQ weights remain available on the [`pre-dwq`](https://huggingface.co/avlp12/GLM-5.2-Alis-MLX-Dynamic-2.56bpw/tree/pre-dwq) branch. **The *Quality* and *Benchmarks* figures below are measured on these DWQ weights** — the retune also moves perplexity (strided wikitext 4.34 → 3.77, code 2.20 → 2.07; `mlx_lm.perplexity` on tulu-3 3.85 → 3.56), not just the KL above.
+Weights history: original quant on [`pre-dwq`](https://huggingface.co/avlp12/GLM-5.2-Alis-MLX-Dynamic-2.56bpw/tree/pre-dwq) · first DWQ retune on [`dwq-noclip`](https://huggingface.co/avlp12/GLM-5.2-Alis-MLX-Dynamic-2.56bpw/tree/dwq-noclip) · current `main` = clip + DWQ.
 
 ## Metrics
 
@@ -44,7 +46,7 @@ The pre-DWQ weights remain available on the [`pre-dwq`](https://huggingface.co/a
 |---|---|
 | Base model | zai-org/GLM-5.2 (744B total / \~40B active) |
 | Bits/weight | **\~2.56** (per-tensor mixed) |
-| On-disk size | **246.9 GB** (48 shards, incl. the 4.5 GB native MTP layer) |
+| On-disk size | **242.4 GB** (47 shards: 46 base + the 4.5 GB native MTP layer, which no longer ships twice) |
 | Peak memory | \~242 GB (weights, MTP-attached) · \~249 GB (8K) · \~263 GB (26K) · \~293 GB (64K) · \~344 GB (128K) — **measured** (int8 KV); the DSA prefill activation grows with context |
 | Format | MLX (Apple Silicon) |
 | Context | 1M-capable architecture (DSA); **machine-limited in practice** — ≈26–32K prefill on a 256 GiB box (see *Long context & memory*) |
@@ -58,7 +60,7 @@ GLM-5.2 is a frontier agentic-coding MoE, but at 744B it is \~1.5 TB in bf16 —
 
 ## Quality
 
-This is the **≤256 GB** option — the routed experts are 2-bit, so it is deliberately bit-starved. If you have a 512 GB machine, the **[3.5 bpw build](https://huggingface.co/avlp12/GLM-5.2-Alis-MLX-Dynamic-3.5bpw)** is still better on prose/code strided PPL (−25% wikitext, −11% code; the 3.5 bpw sibling is DWQ-retuned against an 8-bit teacher) and runs a full 1M context.
+This is the **≤256 GB** option — the routed experts are 2-bit, so it is deliberately bit-starved. If you have a 512 GB machine, the **[3.5 bpw build](https://huggingface.co/avlp12/GLM-5.2-Alis-MLX-Dynamic-3.5bpw)** is still better on prose/code strided PPL (−25% wikitext, −11% code; both builds are clip+DWQ retunes, the 3.5 bpw sibling against an 8-bit teacher) and runs a full 1M context.
 
 ![Perplexity: this 2.56 bpw build vs the 3.5 bpw build — 2.56 bpw is 25% higher (worse) on wikitext, 11% on code](assets/quality.png)
 
@@ -70,13 +72,13 @@ Reproduced with `mlx_lm.evaluate` (0-shot) and `mlx_lm.perplexity` (seq 2048, 50
 
 | | GLM-5.1 · 2.7 bpw | **GLM-5.2 · 2.56 bpw (this)** | GLM-5.2 · 3.5 bpw |
 |---|---|---|---|
-| Perplexity (lower) | 4.165 | **3.564** | 3.630 |
-| HellaSwag (acc_norm) | 0.606 | **0.652** | 0.604 |
-| PIQA (acc) | 0.796 | **0.808** | 0.822 |
+| Perplexity (lower) | 4.165 | **3.571** | 3.644 |
+| HellaSwag (acc_norm) | 0.606 | **0.652** | 0.626 |
+| PIQA (acc) | 0.796 | **0.808** | 0.838 |
 | WinoGrande (acc) | 0.660 | **0.780** | 0.780 |
 | Generation (tok/s) | 18.35 | **22.87** | 21.29 |
 
-Perplexity here is on `allenai/tulu-3-sft-mixture` (the `mlx_lm.perplexity` default) — a different corpus and method from the wikitext strided figure above, so values are not comparable across the two. Task accuracies use a 500-sample limit (CI ±0.02–0.04). GLM-5.1 is a different (older) base model, so cross-generation gaps reflect both the newer model and quantization.
+Perplexity here is on `allenai/tulu-3-sft-mixture` (the `mlx_lm.perplexity` default) — a different corpus and method from the wikitext strided figure above, so values are not comparable across the two. Task accuracies use a 500-sample limit (CI ±0.02–0.04). This column's perplexity is measured on the current `main` (clip+DWQ); its task accuracies were measured on the previous retune (now `dwq-noclip`) — a task-accuracy re-run on the current `main` is queued. The 3.5 bpw column is that build's current `main` (all re-measured). GLM-5.1 is a different (older) base model, so cross-generation gaps reflect both the newer model and quantization.
 
 **Quantization recipe**
 
diff --git a/assets/quality.png b/assets/quality.png
index c416f957b85aeecdf062fc9a04a2bd940219ed97..4bc509c5fc10724bb226c063e4d2f8f2874b9c79 100644
--- a/assets/quality.png
+++ b/assets/quality.png
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1d40d38568fd965753b35d0008be5480e1b68f4ad1e33353bc99a762ef25b705
-size 65854
+oid sha256:9772e208c8728d3d5305163375e4fce56a08ea48b1934bcfaf220fb279891032
+size 61741
diff --git a/assets/quality.svg b/assets/quality.svg
index 23ce1f791002350e44ecd0ad6182ad52ec138ff2..eb1e07db2810acc628b58f7723a4321b4367e0a6 100644
--- a/assets/quality.svg
+++ b/assets/quality.svg
@@ -18,18 +18,18 @@
   <line x1="90" y1="350" x2="650" y2="350" stroke="#cbd5e1"/>
 
   <!-- wikitext group -->
-  <rect x="156" y="161" width="64" height="189" rx="4" fill="#cbd5e1"/>
-  <text x="188" y="154" font-size="13" font-weight="600" fill="#475569" text-anchor="middle">3.774</text>
-  <rect x="232" y="209" width="64" height="141" rx="4" fill="#6366f1"/>
-  <text x="264" y="202" font-size="13" font-weight="600" fill="#4338ca" text-anchor="middle">2.814</text>
+  <rect x="156" y="165" width="64" height="185" rx="4" fill="#cbd5e1"/>
+  <text x="188" y="158" font-size="13" font-weight="600" fill="#475569" text-anchor="middle">3.698</text>
+  <rect x="232" y="211" width="64" height="139" rx="4" fill="#6366f1"/>
+  <text x="264" y="204" font-size="13" font-weight="600" fill="#4338ca" text-anchor="middle">2.777</text>
   <text x="226" y="374" font-size="14" fill="#0f172a" text-anchor="middle">wikitext (prose)</text>
   <text x="264" y="396" font-size="13" font-weight="700" fill="#16a34a" text-anchor="middle">−25%</text>
 
   <!-- code group -->
   <rect x="436" y="247" width="64" height="103" rx="4" fill="#cbd5e1"/>
-  <text x="468" y="240" font-size="13" font-weight="600" fill="#475569" text-anchor="middle">2.069</text>
+  <text x="468" y="240" font-size="13" font-weight="600" fill="#475569" text-anchor="middle">2.054</text>
   <rect x="512" y="258" width="64" height="92" rx="4" fill="#6366f1"/>
-  <text x="544" y="251" font-size="13" font-weight="600" fill="#4338ca" text-anchor="middle">1.832</text>
+  <text x="544" y="251" font-size="13" font-weight="600" fill="#4338ca" text-anchor="middle">1.835</text>
   <text x="506" y="374" font-size="14" fill="#0f172a" text-anchor="middle">code (mlx_lm src)</text>
   <text x="544" y="396" font-size="13" font-weight="700" fill="#16a34a" text-anchor="middle">−11%</text>
 </svg>
diff --git a/config.json b/config.json
index e1724054cf60eb8546d8bb214ffddbd8719d74b9..3f0383dc0d1410520d5fc3df42ba671eb02c6d3a 100644
--- a/config.json
+++ b/config.json
@@ -1,9621 +1,9621 @@
 {
-  "architectures": [
-    "GlmMoeDsaForCausalLM"
-  ],
-  "attention_bias": false,
-  "attention_dropout": 0.0,
-  "dtype": "bfloat16",
-  "eos_token_id": [
-    154820,
-    154827,
-    154829
-  ],
-  "ep_size": 1,
-  "first_k_dense_replace": 3,
-  "head_dim": 192,
-  "hidden_act": "silu",
-  "hidden_size": 6144,
-  "index_head_dim": 128,
-  "index_n_heads": 32,
-  "index_share_for_mtp_iteration": true,
-  "index_skip_topk_offset": 3,
-  "index_topk": 2048,
-  "index_topk_freq": 4,
-  "index_topk_pattern": null,
-  "indexer_norm_eps": 1e-06,
-  "indexer_rope_interleave": true,
-  "indexer_rope_traditional": false,
-  "indexer_types": [
-    "full",
-    "full",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared",
-    "full",
-    "shared",
-    "shared",
-    "shared"
-  ],
-  "initializer_range": 0.02,
-  "intermediate_size": 12288,
-  "kv_lora_rank": 512,
-  "max_position_embeddings": 1048576,
-  "mlp_layer_types": [
-    "dense",
-    "dense",
-    "dense",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse",
-    "sparse"
-  ],
-  "model_type": "glm_moe_dsa",
-  "moe_intermediate_size": 2048,
-  "moe_layer_freq": 1,
-  "n_group": 1,
-  "n_routed_experts": 256,
-  "n_shared_experts": 1,
-  "norm_topk_prob": true,
-  "num_attention_heads": 64,
-  "num_experts_per_tok": 8,
-  "num_hidden_layers": 78,
-  "num_key_value_heads": 64,
-  "num_nextn_predict_layers": 1,
-  "pad_token_id": 154820,
-  "pretraining_tp": 1,
-  "q_lora_rank": 2048,
-  "qk_head_dim": 256,
-  "qk_nope_head_dim": 192,
-  "qk_rope_head_dim": 64,
-  "quantization": {
-    "group_size": 64,
-    "bits": 4,
-    "mode": "affine",
-    "model.embed_tokens": {
-      "group_size": 64,
-      "bits": 6,
-      "mode": "affine"
-    },
-    "model.layers.0.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.0.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.0.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.0.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.0.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.0.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.0.mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.0.mlp.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.0.mlp.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.mlp.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.mlp.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.mlp.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.mlp.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.3.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.3.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.3.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.4.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.4.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.4.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.5.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.5.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.5.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.6.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.6.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.6.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.7.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.7.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.7.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.8.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.8.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.8.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.9.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.9.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.9.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.10.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.10.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.10.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.11.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.11.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.11.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.12.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.12.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.12.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.13.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.13.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.13.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.14.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.14.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.14.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.15.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.15.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.15.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.16.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.16.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.16.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.17.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.17.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.17.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.18.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.18.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.18.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.19.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.19.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.19.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.20.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.20.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.20.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.21.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.21.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.21.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.22.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.22.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.22.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.23.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.23.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.23.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.24.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.24.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.24.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.25.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.25.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.25.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.26.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.26.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.26.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.27.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.27.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.27.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.28.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.28.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.28.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.29.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.29.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.29.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.30.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.30.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.30.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.31.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.31.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.31.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.32.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.32.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.32.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.33.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.33.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.33.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.34.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.34.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.34.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.35.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.35.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.35.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.36.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.36.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.36.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.37.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.37.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.37.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.38.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.38.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.38.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.39.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.39.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.39.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.40.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.40.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.40.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.41.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.41.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.41.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.42.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.42.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.42.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.43.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.43.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.43.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.44.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.44.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.44.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.45.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.45.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.45.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.46.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.46.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.46.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.47.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.47.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.47.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.48.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.48.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.48.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.49.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.49.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.49.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.50.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.50.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.50.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.51.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.51.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.51.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.52.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.52.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.52.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.53.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.53.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.53.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.54.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.54.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.54.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.55.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.55.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.55.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.56.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.56.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.56.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.57.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.57.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.57.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.58.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.58.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.58.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.59.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.59.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.59.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.60.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.60.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.60.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.61.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.61.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.61.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.62.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.62.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.62.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.63.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.63.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.63.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.64.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.64.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.64.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.65.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.65.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.65.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.66.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.66.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.66.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.67.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.67.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.67.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.68.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.68.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.68.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.69.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.69.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.69.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.70.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.70.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.70.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.71.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.71.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.71.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.72.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.72.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.72.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.73.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.73.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.73.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.74.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.74.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.74.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.75.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.75.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.75.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.76.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.76.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.76.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.77.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.77.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.77.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "lm_head": {
-      "group_size": 64,
-      "bits": 6,
-      "mode": "affine"
-    },
-    "mtp.layer.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 3
-    },
-    "mtp.layer.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4
-    },
-    "mtp.layer.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 3
-    },
-    "mtp.layer.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4
-    },
-    "mtp.layer.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 3
-    },
-    "mtp.layer.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4
-    },
-    "mtp.layer.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4
-    },
-    "mtp.layer.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4
-    },
-    "mtp.layer.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4
-    },
-    "mtp.layer.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4
-    },
-    "mtp.layer.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4
-    },
-    "mtp.layer.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4
-    }
-  },
-  "quantization_config": {
-    "group_size": 64,
-    "bits": 4,
-    "mode": "affine",
-    "model.embed_tokens": {
-      "group_size": 64,
-      "bits": 6,
-      "mode": "affine"
-    },
-    "model.layers.0.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.0.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.0.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.0.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.0.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.0.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.0.mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.0.mlp.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.0.mlp.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.mlp.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.1.mlp.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.mlp.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.2.mlp.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.3.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.3.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.3.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.3.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.4.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.4.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.4.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.4.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.5.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.5.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.5.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.5.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.6.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.6.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.6.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.6.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.7.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.7.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.7.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.7.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.8.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.8.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.8.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.8.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.9.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.9.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.9.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.9.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.10.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.10.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.10.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.10.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.11.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.11.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.11.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.11.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.12.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.12.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.12.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.12.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.13.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.13.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.13.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.13.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.14.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.14.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.14.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.14.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.15.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.15.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.15.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.15.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.16.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.16.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.16.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.16.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.17.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.17.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.17.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.17.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.18.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.18.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.18.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.18.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.19.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.19.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.19.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.19.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.20.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.20.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.20.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.20.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.21.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.21.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.21.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.21.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.22.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.22.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.22.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.22.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.23.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.23.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.23.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.23.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.24.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.24.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.24.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.24.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.25.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.25.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.25.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.25.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.26.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.26.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.26.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.26.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.27.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.27.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.27.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.27.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.28.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.28.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.28.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.28.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.29.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.29.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.29.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.29.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.30.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.30.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.30.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.30.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.31.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.31.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.31.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.31.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.32.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.32.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.32.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.32.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.33.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.33.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.33.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.33.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.34.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.34.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.34.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.34.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.35.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.35.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.35.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.35.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.36.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.36.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.36.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.36.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.37.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.37.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.37.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.37.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.38.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.38.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.38.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.38.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.39.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.39.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.39.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.39.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.40.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.40.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.40.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.40.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.41.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.41.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.41.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.41.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.42.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.42.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.42.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.42.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.43.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.43.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.43.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.43.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.44.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.44.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.44.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.44.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.45.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.45.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.45.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.45.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.46.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.46.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.46.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.46.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.47.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.47.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.47.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.47.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.48.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.48.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.48.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.48.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.49.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.49.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.49.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.49.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.50.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.50.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.50.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.50.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.51.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.51.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.51.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.51.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.52.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.52.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.52.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.52.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.53.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.53.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.53.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.53.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.54.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.54.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.54.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.54.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.55.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.55.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.55.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.55.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.56.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.56.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.56.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.56.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.57.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.57.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.57.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.57.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.58.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.58.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.58.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.58.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.59.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.59.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.59.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.59.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.60.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.60.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.60.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.60.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.61.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.61.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.61.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.61.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.62.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.62.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.62.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.62.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.63.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.63.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.63.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.63.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.64.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.64.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.64.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.64.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.65.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.65.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.65.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.65.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.66.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.66.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.66.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.66.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.67.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.67.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.67.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.67.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.68.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.68.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.68.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.68.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.69.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.69.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.69.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.69.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.70.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.70.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.70.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.70.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.71.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.71.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.71.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.71.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.72.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.72.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.72.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.72.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.73.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.73.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.73.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.73.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.74.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.74.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.74.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.74.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.75.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.75.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.75.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.75.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.76.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.76.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.76.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.76.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.77.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.77.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 2,
-      "mode": "affine"
-    },
-    "model.layers.77.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "model.layers.77.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4,
-      "mode": "affine"
-    },
-    "lm_head": {
-      "group_size": 64,
-      "bits": 6,
-      "mode": "affine"
-    },
-    "mtp.layer.mlp.switch_mlp.gate_proj": {
-      "group_size": 64,
-      "bits": 3
-    },
-    "mtp.layer.mlp.shared_experts.gate_proj": {
-      "group_size": 64,
-      "bits": 4
-    },
-    "mtp.layer.mlp.switch_mlp.down_proj": {
-      "group_size": 64,
-      "bits": 3
-    },
-    "mtp.layer.mlp.shared_experts.down_proj": {
-      "group_size": 64,
-      "bits": 4
-    },
-    "mtp.layer.mlp.switch_mlp.up_proj": {
-      "group_size": 64,
-      "bits": 3
-    },
-    "mtp.layer.mlp.shared_experts.up_proj": {
-      "group_size": 64,
-      "bits": 4
-    },
-    "mtp.layer.self_attn.q_a_proj": {
-      "group_size": 64,
-      "bits": 4
-    },
-    "mtp.layer.self_attn.q_b_proj": {
-      "group_size": 64,
-      "bits": 4
-    },
-    "mtp.layer.self_attn.kv_a_proj_with_mqa": {
-      "group_size": 64,
-      "bits": 4
-    },
-    "mtp.layer.self_attn.o_proj": {
-      "group_size": 64,
-      "bits": 4
-    },
-    "mtp.layer.self_attn.embed_q": {
-      "group_size": 64,
-      "bits": 4
-    },
-    "mtp.layer.self_attn.unembed_out": {
-      "group_size": 64,
-      "bits": 4
-    }
-  },
-  "rms_norm_eps": 1e-05,
-  "rope_interleave": true,
-  "rope_parameters": {
-    "rope_theta": 8000000,
-    "rope_type": "default"
-  },
-  "routed_scaling_factor": 2.5,
-  "scoring_func": "sigmoid",
-  "tie_word_embeddings": false,
-  "topk_group": 1,
-  "topk_method": "noaux_tc",
-  "transformers_version": "5.12.0",
-  "use_cache": true,
-  "v_head_dim": 256,
-  "vocab_size": 154880
+    "architectures": [
+        "GlmMoeDsaForCausalLM"
+    ],
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "dtype": "bfloat16",
+    "eos_token_id": [
+        154820,
+        154827,
+        154829
+    ],
+    "ep_size": 1,
+    "first_k_dense_replace": 3,
+    "head_dim": 192,
+    "hidden_act": "silu",
+    "hidden_size": 6144,
+    "index_head_dim": 128,
+    "index_n_heads": 32,
+    "index_share_for_mtp_iteration": true,
+    "index_skip_topk_offset": 3,
+    "index_topk": 2048,
+    "index_topk_freq": 4,
+    "index_topk_pattern": null,
+    "indexer_norm_eps": 1e-06,
+    "indexer_rope_interleave": true,
+    "indexer_rope_traditional": false,
+    "indexer_types": [
+        "full",
+        "full",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared",
+        "full",
+        "shared",
+        "shared",
+        "shared"
+    ],
+    "initializer_range": 0.02,
+    "intermediate_size": 12288,
+    "kv_lora_rank": 512,
+    "max_position_embeddings": 1048576,
+    "mlp_layer_types": [
+        "dense",
+        "dense",
+        "dense",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse",
+        "sparse"
+    ],
+    "model_type": "glm_moe_dsa",
+    "moe_intermediate_size": 2048,
+    "moe_layer_freq": 1,
+    "n_group": 1,
+    "n_routed_experts": 256,
+    "n_shared_experts": 1,
+    "norm_topk_prob": true,
+    "num_attention_heads": 64,
+    "num_experts_per_tok": 8,
+    "num_hidden_layers": 78,
+    "num_key_value_heads": 64,
+    "num_nextn_predict_layers": 1,
+    "pad_token_id": 154820,
+    "pretraining_tp": 1,
+    "q_lora_rank": 2048,
+    "qk_head_dim": 256,
+    "qk_nope_head_dim": 192,
+    "qk_rope_head_dim": 64,
+    "quantization": {
+        "group_size": 64,
+        "bits": 4,
+        "mode": "affine",
+        "model.embed_tokens": {
+            "group_size": 64,
+            "bits": 6,
+            "mode": "affine"
+        },
+        "model.layers.0.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.0.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.0.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.0.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.0.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.0.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.0.mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.0.mlp.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.0.mlp.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.mlp.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.mlp.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.mlp.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.mlp.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.3.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.3.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.3.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.4.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.4.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.4.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.5.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.5.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.5.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.6.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.6.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.6.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.7.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.7.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.7.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.8.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.8.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.8.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.9.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.9.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.9.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.10.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.10.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.10.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.11.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.11.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.11.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.12.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.12.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.12.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.13.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.13.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.13.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.14.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.14.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.14.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.15.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.15.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.15.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.16.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.16.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.16.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.17.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.17.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.17.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.18.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.18.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.18.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.19.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.19.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.19.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.20.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.20.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.20.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.21.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.21.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.21.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.22.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.22.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.22.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.23.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.23.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.23.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.24.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.24.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.24.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.25.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.25.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.25.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.26.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.26.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.26.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.27.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.27.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.27.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.28.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.28.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.28.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.29.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.29.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.29.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.30.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.30.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.30.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.31.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.31.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.31.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.32.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.32.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.32.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.33.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.33.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.33.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.34.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.34.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.34.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.35.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.35.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.35.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.36.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.36.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.36.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.37.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.37.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.37.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.38.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.38.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.38.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.39.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.39.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.39.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.40.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.40.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.40.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.41.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.41.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.41.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.42.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.42.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.42.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.43.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.43.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.43.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.44.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.44.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.44.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.45.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.45.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.45.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.46.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.46.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.46.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.47.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.47.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.47.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.48.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.48.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.48.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.49.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.49.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.49.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.50.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.50.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.50.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.51.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.51.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.51.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.52.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.52.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.52.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.53.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.53.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.53.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.54.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.54.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.54.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.55.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.55.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.55.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.56.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.56.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.56.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.57.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.57.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.57.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.58.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.58.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.58.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.59.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.59.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.59.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.60.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.60.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.60.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.61.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.61.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.61.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.62.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.62.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.62.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.63.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.63.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.63.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.64.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.64.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.64.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.65.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.65.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.65.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.66.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.66.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.66.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.67.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.67.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.67.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.68.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.68.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.68.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.69.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.69.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.69.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.70.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.70.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.70.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.71.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.71.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.71.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.72.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.72.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.72.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.73.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.73.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.73.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.74.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.74.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.74.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.75.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.75.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.75.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.76.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.76.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.76.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.77.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.77.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.77.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "lm_head": {
+            "group_size": 64,
+            "bits": 6,
+            "mode": "affine"
+        },
+        "mtp.layer.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 3
+        },
+        "mtp.layer.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4
+        },
+        "mtp.layer.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 3
+        },
+        "mtp.layer.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4
+        },
+        "mtp.layer.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 3
+        },
+        "mtp.layer.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4
+        },
+        "mtp.layer.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4
+        },
+        "mtp.layer.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4
+        },
+        "mtp.layer.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4
+        },
+        "mtp.layer.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4
+        },
+        "mtp.layer.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4
+        },
+        "mtp.layer.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4
+        }
+    },
+    "quantization_config": {
+        "group_size": 64,
+        "bits": 4,
+        "mode": "affine",
+        "model.embed_tokens": {
+            "group_size": 64,
+            "bits": 6,
+            "mode": "affine"
+        },
+        "model.layers.0.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.0.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.0.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.0.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.0.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.0.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.0.mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.0.mlp.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.0.mlp.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.mlp.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.1.mlp.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.mlp.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.2.mlp.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.3.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.3.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.3.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.3.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.4.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.4.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.4.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.4.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.5.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.5.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.5.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.5.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.6.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.6.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.6.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.6.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.7.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.7.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.7.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.7.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.8.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.8.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.8.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.8.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.9.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.9.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.9.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.9.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.10.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.10.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.10.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.10.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.11.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.11.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.11.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.11.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.12.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.12.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.12.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.12.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.13.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.13.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.13.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.13.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.14.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.14.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.14.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.14.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.15.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.15.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.15.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.15.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.16.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.16.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.16.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.16.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.17.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.17.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.17.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.17.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.18.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.18.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.18.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.18.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.19.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.19.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.19.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.19.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.20.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.20.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.20.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.20.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.21.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.21.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.21.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.21.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.22.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.22.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.22.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.22.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.23.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.23.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.23.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.23.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.24.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.24.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.24.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.24.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.25.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.25.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.25.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.25.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.26.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.26.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.26.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.26.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.27.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.27.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.27.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.27.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.28.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.28.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.28.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.28.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.29.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.29.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.29.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.29.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.30.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.30.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.30.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.30.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.31.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.31.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.31.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.31.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.32.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.32.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.32.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.32.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.33.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.33.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.33.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.33.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.34.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.34.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.34.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.34.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.35.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.35.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.35.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.35.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.36.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.36.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.36.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.36.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.37.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.37.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.37.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.37.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.38.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.38.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.38.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.38.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.39.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.39.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.39.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.39.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.40.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.40.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.40.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.40.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.41.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.41.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.41.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.41.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.42.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.42.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.42.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.42.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.43.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.43.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.43.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.43.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.44.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.44.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.44.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.44.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.45.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.45.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.45.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.45.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.46.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.46.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.46.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.46.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.47.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.47.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.47.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.47.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.48.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.48.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.48.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.48.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.49.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.49.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.49.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.49.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.50.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.50.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.50.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.50.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.51.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.51.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.51.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.51.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.52.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.52.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.52.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.52.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.53.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.53.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.53.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.53.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.54.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.54.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.54.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.54.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.55.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.55.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.55.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.55.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.56.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.56.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.56.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.56.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.57.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.57.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.57.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.57.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.58.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.58.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.58.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.58.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.59.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.59.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.59.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.59.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.60.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.60.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.60.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.60.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.61.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.61.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.61.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.61.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.62.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.62.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.62.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.62.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.63.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.63.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.63.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.63.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.64.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.64.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.64.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.64.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.65.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.65.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.65.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.65.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.66.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.66.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.66.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.66.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.67.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.67.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.67.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.67.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.68.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.68.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.68.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.68.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.69.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.69.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.69.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.69.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.70.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.70.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.70.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.70.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.71.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.71.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.71.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.71.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.72.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.72.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.72.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.72.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.73.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.73.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.73.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.73.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.74.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.74.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.74.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.74.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.75.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.75.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.75.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.75.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.76.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.76.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.76.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.76.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.77.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.77.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 2,
+            "mode": "affine"
+        },
+        "model.layers.77.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "model.layers.77.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4,
+            "mode": "affine"
+        },
+        "lm_head": {
+            "group_size": 64,
+            "bits": 6,
+            "mode": "affine"
+        },
+        "mtp.layer.mlp.switch_mlp.gate_proj": {
+            "group_size": 64,
+            "bits": 3
+        },
+        "mtp.layer.mlp.shared_experts.gate_proj": {
+            "group_size": 64,
+            "bits": 4
+        },
+        "mtp.layer.mlp.switch_mlp.down_proj": {
+            "group_size": 64,
+            "bits": 3
+        },
+        "mtp.layer.mlp.shared_experts.down_proj": {
+            "group_size": 64,
+            "bits": 4
+        },
+        "mtp.layer.mlp.switch_mlp.up_proj": {
+            "group_size": 64,
+            "bits": 3
+        },
+        "mtp.layer.mlp.shared_experts.up_proj": {
+            "group_size": 64,
+            "bits": 4
+        },
+        "mtp.layer.self_attn.q_a_proj": {
+            "group_size": 64,
+            "bits": 4
+        },
+        "mtp.layer.self_attn.q_b_proj": {
+            "group_size": 64,
+            "bits": 4
+        },
+        "mtp.layer.self_attn.kv_a_proj_with_mqa": {
+            "group_size": 64,
+            "bits": 4
+        },
+        "mtp.layer.self_attn.o_proj": {
+            "group_size": 64,
+            "bits": 4
+        },
+        "mtp.layer.self_attn.embed_q": {
+            "group_size": 64,
+            "bits": 4
+        },
+        "mtp.layer.self_attn.unembed_out": {
+            "group_size": 64,
+            "bits": 4
+        }
+    },
+    "rms_norm_eps": 1e-05,
+    "rope_interleave": true,
+    "rope_parameters": {
+        "rope_theta": 8000000,
+        "rope_type": "default"
+    },
+    "routed_scaling_factor": 2.5,
+    "scoring_func": "sigmoid",
+    "tie_word_embeddings": false,
+    "topk_group": 1,
+    "topk_method": "noaux_tc",
+    "transformers_version": "5.12.0",
+    "use_cache": true,
+    "v_head_dim": 256,
+    "vocab_size": 154880
 }
\ No newline at end of file
diff --git a/model-00001-of-00047.safetensors b/model-00001-of-00047.safetensors
index 230b8af46edaa55381344655ef443776d9412100..fb2b2fc8b96e1dd60b4565c6d652c54ad8613655 100644
--- a/model-00001-of-00047.safetensors
+++ b/model-00001-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6a4940f13fa66bbffe0179befc8444ecfcea7298494f2e4c3380aec73245db2b
+oid sha256:004fbaeac7db373cb77541d4cc64cfb448893fba19ef935945c51b7270b189ac
 size 4720139752
diff --git a/model-00002-of-00047.safetensors b/model-00002-of-00047.safetensors
index 664086adcfa280b592a78ec4314f86fcafec990d..136a0819ec9c6f75c92344abe31f478ba6322ba2 100644
--- a/model-00002-of-00047.safetensors
+++ b/model-00002-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2ef903c239e355401ee8fbb14d6a453ca427f8a838ec14893195fb28f6039b21
+oid sha256:175661504973cffd2872c567a1f347cd28ce86030e9b15b44001f168d27a4111
 size 5150404363
diff --git a/model-00003-of-00047.safetensors b/model-00003-of-00047.safetensors
index 4d1f5b52254c350720afaf4098ff64a9fd437330..242fd7d3d2494b498d5fab20c5de0a4f1bccd8f9 100644
--- a/model-00003-of-00047.safetensors
+++ b/model-00003-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0c14f10a1c8a73c411086d677ecadc1f7b40dfa7bc79be36b1138a24466c914c
+oid sha256:44e7f0786ba71b2e9089ebefa1106b7f81a22163c26f6c013a1bf556718d09ef
 size 5286386379
diff --git a/model-00004-of-00047.safetensors b/model-00004-of-00047.safetensors
index 517ca149c169a053282b9f3412404795e3cff49b..69a91baf1c62c8117a9b43b22ebcd7d45e26016e 100644
--- a/model-00004-of-00047.safetensors
+++ b/model-00004-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0ff1e786ce75d6f279e5c777be1b87a89b5bf5c4362d2336e833775e83d61bf1
+oid sha256:951b0790233659e390783fc8928295ee4cdeaad0cbe645b07a9b5c0ca21f3fb4
 size 5267641891
diff --git a/model-00005-of-00047.safetensors b/model-00005-of-00047.safetensors
index 0751be0a550b673ae6e26f5f0f51dd2f31b6becb..8c5853d44da8453b126769dc52b0c06de8f686e8 100644
--- a/model-00005-of-00047.safetensors
+++ b/model-00005-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:907dc6c8b2ad41ed38dd8bbe6ba0097793dd2f8e00eaed8fb860f1538840b992
+oid sha256:65baa849dba3c6a842d8f86dff809e19e8d9098c3330ed2c9e69f7c754bbf357
 size 5169148808
diff --git a/model-00006-of-00047.safetensors b/model-00006-of-00047.safetensors
index 1efd1e537dc1a4c4161ea39732a0be9e7fbf2db2..3369c725eeba6b2fdaf5c0008e6de12f06e27e37 100644
--- a/model-00006-of-00047.safetensors
+++ b/model-00006-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6884f72dd597a89f890621f1462aba69b8b5f184f08586ebe37e361530f9e124
+oid sha256:5b7e75090bf85c987e7e512c1eeb5c61debd67657a2b32ee976b1571980d96d1
 size 5267642066
diff --git a/model-00007-of-00047.safetensors b/model-00007-of-00047.safetensors
index 3115a4ab8e05f7925009fd340897dfd439214562..66c7ea4b4575b291fc95a32a9a14105c88efdc2d 100644
--- a/model-00007-of-00047.safetensors
+++ b/model-00007-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5d57f5103595f98767f223ed314ac50bbc6ab7efd629b6cc19946482f2bc462a
+oid sha256:07f1feeae8cfb9d8a464f0c9be3479da80f90a7e5e5283a1e6092532c4c99fa8
 size 5286386413
diff --git a/model-00008-of-00047.safetensors b/model-00008-of-00047.safetensors
index 786998e69261ce1dbef67a994918081d7d4e9031..d67bfd928d8655f76a0f0623e742923ca91ce4e6 100644
--- a/model-00008-of-00047.safetensors
+++ b/model-00008-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:385b518fd74b88b22d5103fa372054ebc4be54ff0c8819ae3ec935698071934c
+oid sha256:9483a9d597cfab5181f583da7f7c1f38f6220456e6d555f4bf294d3f17d89cf1
 size 5150404393
diff --git a/model-00009-of-00047.safetensors b/model-00009-of-00047.safetensors
index 386fb66067f20538037112cbe853256dc86439de..810a1f215596ad8187dc1767a1798025d1ff0084 100644
--- a/model-00009-of-00047.safetensors
+++ b/model-00009-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c7d5c8cac2a8c46b45ee551ecaa2f3f9afd0067ee9bec1de62f857e8142f9a5b
+oid sha256:01f29bbd0f8bedbcaf75396b724a2aed5465a69083efd2ca1b68ebe6014496f0
 size 5267642054
diff --git a/model-00010-of-00047.safetensors b/model-00010-of-00047.safetensors
index dd4a7499cac7fed6d1d0d183293855624161e1b0..ffad1020ceb2953483d60c59d79831a1283b5e59 100644
--- a/model-00010-of-00047.safetensors
+++ b/model-00010-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:afdf9dc23e7938686edf32ed39d253a52b011b569c45a85d23e423993e6519aa
+oid sha256:591f4c3f82ff6238b058d99e236f3a81c48b37716a63a97ba3f9b6485e467093
 size 5286386433
diff --git a/model-00011-of-00047.safetensors b/model-00011-of-00047.safetensors
index fb3adfa48acce38fa4411d918aaa80e08ffd432b..72e5213992f99a411cf760068f0bec3cce8679d4 100644
--- a/model-00011-of-00047.safetensors
+++ b/model-00011-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:61f4967eb1b4da1bb6926917963dd4333281ebfc4218675edb6fa3b8621a11d2
+oid sha256:98967e6de2686d7c3a1ec251b9b2505886010779ec406e9f44495305dc06b681
 size 5150404409
diff --git a/model-00012-of-00047.safetensors b/model-00012-of-00047.safetensors
index bbe3e02acc4bf7b12c6d2ed1ff3e42fec87cf036..ae81f12e411c4b28739ec972842382e2da1eb298 100644
--- a/model-00012-of-00047.safetensors
+++ b/model-00012-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4d6ef0c6c2da27254716846aec19c46102added0d3d0344dba901d38a7fedf39
+oid sha256:f26da6cd9cb630aec4dbe10d4bc0881899d5f9ce7e249fb517fbaca4a5759d6d
 size 5286386459
diff --git a/model-00013-of-00047.safetensors b/model-00013-of-00047.safetensors
index 8ef91a0274feb1f350d115eb57c8ff2ef7c40176..a3b19e344719141a929a075c5ff6b031674c6c68 100644
--- a/model-00013-of-00047.safetensors
+++ b/model-00013-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f2a160455286773ac2bb1b3539512b8819c5fd3efcc6ee108e7fd15eda374985
+oid sha256:ed93032925e3d6fb2fd1b066a87d21b30eb3b339f4286b4840f13f7ee46e2c59
 size 5267642000
diff --git a/model-00014-of-00047.safetensors b/model-00014-of-00047.safetensors
index dd07ee8559b3c65204e6dab67a74f36b3b98893d..fc84a6356c49512554f1de23065eb288bcaf9a81 100644
--- a/model-00014-of-00047.safetensors
+++ b/model-00014-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1488417083b95c8dea6fb687d31f31ea55b530441e75437cc94d3b3d07210ec8
+oid sha256:9206a05b46e4d387b78dfbf2318dc06a644c7905837b2bec368ef61c97d9c7e3
 size 5150404409
diff --git a/model-00015-of-00047.safetensors b/model-00015-of-00047.safetensors
index 17669d7a33b92e403002445ea7dceb3953fc460d..19ab98e4f7bf66591f800fbed8c0f303a3a57d0e 100644
--- a/model-00015-of-00047.safetensors
+++ b/model-00015-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0de16cae33d6dc83ca220175225144b1432492ede39448dda4163850a489d1e7
+oid sha256:d562a05059f522f1c72adf43d7b6e0ecf074f26f03d4c1289fc37f81e8bc7afa
 size 5286386493
diff --git a/model-00016-of-00047.safetensors b/model-00016-of-00047.safetensors
index c4f1fdbe852190e8373caa3e3261732c79f755ed..fac8a61782a7823122af899ecaecde15c56861c4 100644
--- a/model-00016-of-00047.safetensors
+++ b/model-00016-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:87357d30246b39e09423c6d8cbb4b3973a05ad3147b73fe8719a008f55382216
+oid sha256:00aadafc65250d712bdc9efef751c9c57869fc3a56b58a91192b5eba20214aee
 size 5267641968
diff --git a/model-00017-of-00047.safetensors b/model-00017-of-00047.safetensors
index dad2d3a655692141ced216e9511d1aa727430dba..3849f3217c6ff39eb605a9976d06d8b919996804 100644
--- a/model-00017-of-00047.safetensors
+++ b/model-00017-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:09d0c1b5f980c3c3136fde813c2484d4f5aa8f202659916b569538010e5be025
+oid sha256:b0bb3ed42952336b68375de98c212e5c6a291534e58d1c9e69411b2359e393f7
 size 5169148832
diff --git a/model-00018-of-00047.safetensors b/model-00018-of-00047.safetensors
index d7e40a255d43f67fda9e7ea289a14d0aca586b2a..eec5d354c0befef37ed7a07d54f672507b1d706e 100644
--- a/model-00018-of-00047.safetensors
+++ b/model-00018-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:68eea5b9b58699365f6d2e5c24050a143cce4baab7c27d4782df38c72132b61e
+oid sha256:eb67d4bd91bfa408b5c23e1b4fa8731c961c9c5f85efea6067ec97d9313e53d8
 size 5267642050
diff --git a/model-00019-of-00047.safetensors b/model-00019-of-00047.safetensors
index 1b3dea96ecfb9a4d4256ac0e1412273b090d755f..436dce46d092c5ceb539f582aeaaa1cc1e2a1d9d 100644
--- a/model-00019-of-00047.safetensors
+++ b/model-00019-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7d2b0fd9a1868e56833a793c83e0ca0ff38282ff84a7c821a5a6bde2931e6256
+oid sha256:6a6288778eadd867b499b60ec39d7dad4d6552e0f144574773b1f6774b21771e
 size 5286386411
diff --git a/model-00020-of-00047.safetensors b/model-00020-of-00047.safetensors
index d8e43134d6558bf80958f78488487543b9841b26..3a1c6bfc6cf6308b134f09c05feb1ab8b937bc8b 100644
--- a/model-00020-of-00047.safetensors
+++ b/model-00020-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ac7ebe9c2ccf446f034fcb8bf4bef2a4291ffecc884b64238988c1cdfaee9232
+oid sha256:22ed9fc05d50e00986f9681cf0a22a7f6c0f8155e72e2c06f4fbc1d001a6187b
 size 5150404399
diff --git a/model-00021-of-00047.safetensors b/model-00021-of-00047.safetensors
index 308fb75ed60dc16896f218d6f077ec736ad46cce..ddbdb92b8d1e409249e16a9baf9c2eb7b6fc710d 100644
--- a/model-00021-of-00047.safetensors
+++ b/model-00021-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:27da782a7bff0780010e550c96e6feec86577ece859988544ed71a76793f7b9e
+oid sha256:b1638dd9480cfdb9dbe4072e01d52ee421fe39c59df2424b562e7743b3d27e8c
 size 5267642036
diff --git a/model-00022-of-00047.safetensors b/model-00022-of-00047.safetensors
index 3717c0490f0920360f5b302d55c2f03eee68ef43..44e20d55f163f6a47761134049255ca07f21b7d5 100644
--- a/model-00022-of-00047.safetensors
+++ b/model-00022-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:320aac92d2b229b76299865045c5ca46d6195dd87c90030943a243d7581f972a
+oid sha256:abdafa0a5568b06f7f5b0d8aab16dfb04c99bf5f73f7f8a562cab90628224304
 size 5286386421
diff --git a/model-00023-of-00047.safetensors b/model-00023-of-00047.safetensors
index d6d0ff5bb752c8a17bdc3aa98e46dce9cce86cbe..bfe6530560d9a0fef0b7944dff32f43cee2bfc02 100644
--- a/model-00023-of-00047.safetensors
+++ b/model-00023-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6e481ba6f816f34476af50bbb03be1e3a2074d813eacb8de51e158e94d7c7598
+oid sha256:3e46e54856a010b97aac321f77c18718b78f7dbc955a9ee877e06f8701a70a67
 size 5150404381
diff --git a/model-00024-of-00047.safetensors b/model-00024-of-00047.safetensors
index a03580644e4545e14b20820164d3473b4a4da6cb..1f3725f9b7733898c4580e7e4a6f064226c4473b 100644
--- a/model-00024-of-00047.safetensors
+++ b/model-00024-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:618eaadcb23d42b61e4288383695caebba37a47dd737ae7d97ff5c25708c8b9e
+oid sha256:10d40fb04b895ea04f703bd132ac3f380cdf3aa336366100b2da0b24be1d0a1e
 size 5286386447
diff --git a/model-00025-of-00047.safetensors b/model-00025-of-00047.safetensors
index dd944c1258115cc69b7e19f629f6fcb1dec1e824..5d04a35f0ebe0575d8a6cce9a8c6f8d5d60ea6ab 100644
--- a/model-00025-of-00047.safetensors
+++ b/model-00025-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:49f5b2b62de98a0cd4a5aed3c3d2c65ef7baf042e675e578959dff80a06232e4
+oid sha256:70a6f9e50b16bb86813f9fd8982398f925d61865dcc5b72b2ac482d6467b3932
 size 5267642004
diff --git a/model-00026-of-00047.safetensors b/model-00026-of-00047.safetensors
index 15136506783ddd4890a2000b31ac4aae6eccc61d..ce463cc28149a7afe07cbdd9895708f5504af1c2 100644
--- a/model-00026-of-00047.safetensors
+++ b/model-00026-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:29e363694db8810e75bf07d4f14ca3e3d8fc2f5c13bd19d0d98771be4133cdd4
+oid sha256:a3664a3420aafd9f76f357f015c6d1ac899f34a7d36328d6f7a1a734c181db9d
 size 5150404401
diff --git a/model-00027-of-00047.safetensors b/model-00027-of-00047.safetensors
index 732942d1a6e17ba29d6555ba99abac30bc4c582f..7190ad9603b38f721ac176ba91dcf27df6b4f342 100644
--- a/model-00027-of-00047.safetensors
+++ b/model-00027-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:df1b0f9e2eb88af211ece851517d76c8ed21a48155f2931502be74b032cdd863
+oid sha256:2054385a09f60949885a25f14788aba56148039ea443443967409c4c0e9a9871
 size 5286386459
diff --git a/model-00028-of-00047.safetensors b/model-00028-of-00047.safetensors
index e7a99564be3c81d18c3471cd66448675dc8af989..82a4a355a520284ffbdaed0068f23e25d77bc0b3 100644
--- a/model-00028-of-00047.safetensors
+++ b/model-00028-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cc72d25c2045861b1400ec00c82755f915ca5e1fd70c8c3d666f259962d95f45
+oid sha256:255510c734a911b453238484633235b319c0e35509c48c68f3f30c647d66e037
 size 5267641994
diff --git a/model-00029-of-00047.safetensors b/model-00029-of-00047.safetensors
index 92ab36e52770fcfabaf30047657512b99da5625c..c1ffbd7364bcbda4afe79fbc57997c664f6ec6ab 100644
--- a/model-00029-of-00047.safetensors
+++ b/model-00029-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d244ce6ecb998da15a9a88b986d3664093616b0090f46420c1a65d18d7483318
+oid sha256:4ed4ed5614c8884df5e377db66b52898eefb41834a94226e179db837642e63a3
 size 5169148832
diff --git a/model-00030-of-00047.safetensors b/model-00030-of-00047.safetensors
index f1bc74d52ed7aa082f36412b78e8f01aba8e8ed8..c2847ee1a48f03a09412adc1ea06908ccb4acd92 100644
--- a/model-00030-of-00047.safetensors
+++ b/model-00030-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3c90a69358a0f6f40ef860ab48416b541068a45dd1934ad267a9182ed7baa064
+oid sha256:213226a26b7928b5f23afc81016cf21699c15e264ee75a73728495c6ec97014a
 size 5267642066
diff --git a/model-00031-of-00047.safetensors b/model-00031-of-00047.safetensors
index 19b63c974a07de48276239c301c2bfc2b2d442a7..83669b10da2978ea9aaeac7f573168d32ac0412e 100644
--- a/model-00031-of-00047.safetensors
+++ b/model-00031-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ffc9e60858cabbd05d2d92fe4bb676ba458bf71f61442191ac33818be347c263
+oid sha256:ef5eead9583a6f1c49cc92ca8caa43abc304337ec0c18440a2fcb011fd359ca1
 size 5286386417
diff --git a/model-00032-of-00047.safetensors b/model-00032-of-00047.safetensors
index 03bf7d0e5edbe786a715aedda2a2255be7254c72..1b67979250a5ca278f34a120a57133d19a8e7ee9 100644
--- a/model-00032-of-00047.safetensors
+++ b/model-00032-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:002ff572ea3e7432ff2676add4d6e42e6ee45225743ec8013db031b79c16d292
+oid sha256:d69193f6abd05ff495f9d1f6ae8dc2daddefcede2369636f0fec1ff19ecf26fd
 size 5150404399
diff --git a/model-00033-of-00047.safetensors b/model-00033-of-00047.safetensors
index caf53b92251daa3c0af5749715bf94bde3aaa7fe..dfde20aa497c7f9819e288e9405ede86c31621c5 100644
--- a/model-00033-of-00047.safetensors
+++ b/model-00033-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:90fec08ba906faa9ad37ad947f0a8da2a1f86ec6f3b248b75ac84832ee3aecbb
+oid sha256:79b6275b85808bb84749ee8b1d3be5853ca1680e0f1433e78b412ee7c0173afd
 size 5267642052
diff --git a/model-00034-of-00047.safetensors b/model-00034-of-00047.safetensors
index 23b49d32692a15421b05b58344bfe4e416388ef1..193f16825e49d9588981f436666957190794b601 100644
--- a/model-00034-of-00047.safetensors
+++ b/model-00034-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:542477f95736743a0ea838828f61c9bffcf35f865e87f79876ddaaa262599a7a
+oid sha256:5959b3d13b7794ee97a68aec73ed1f7e6073126299d032eda534eb0ea988cf1b
 size 5286386423
diff --git a/model-00035-of-00047.safetensors b/model-00035-of-00047.safetensors
index f852bb50d1615c2677f281b2739b605f83f12394..7ca3b70d591f37d7c025b5c196d9c61b852253e1 100644
--- a/model-00035-of-00047.safetensors
+++ b/model-00035-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9076d25e40677647f5df4703e77576c5ed6ae2331dcba6712535ba9242a60d25
+oid sha256:c13f8b0e4ca255f0592f06257b2c43cd3b66a318467d9f40cb4c3072d484872c
 size 5150404411
diff --git a/model-00036-of-00047.safetensors b/model-00036-of-00047.safetensors
index c7f93cd98291224b8f62f48ccb674ba1c31fbf7d..02c375c710f13a98c8c0ad0360f7783c82876e51 100644
--- a/model-00036-of-00047.safetensors
+++ b/model-00036-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ff73977e96ceaa3888710a92e7cae5ad365039c562be00b401ef608d79dd774e
+oid sha256:76831644ce07a658eaf9cf3105a2d4f339633c69321340cb309161a9c1fbd087
 size 5286386467
diff --git a/model-00037-of-00047.safetensors b/model-00037-of-00047.safetensors
index 5172623570c5e344909dce28fb2a1938c2080813..b58d9979dc1c577df5594d27949424ebb3df3e4d 100644
--- a/model-00037-of-00047.safetensors
+++ b/model-00037-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:14879964d8ccb3308a2fda96c57987eb0b67023440e0f426bf997c6841724c47
+oid sha256:d4ff61c9d03f90445c3ae55120105575b774b962ab3fd5cc6b72f51ba752c9c6
 size 5267641972
diff --git a/model-00038-of-00047.safetensors b/model-00038-of-00047.safetensors
index b41ad3dda26b810c112c4876545053eb59942ca3..db25d7035101fe15667adcfddd9b318eb72c3dde 100644
--- a/model-00038-of-00047.safetensors
+++ b/model-00038-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:aa2f6dc99ae8577086c85ba19c00e68d4d130c92a4e60c568f8fdd5ba2ef13cc
+oid sha256:d156e5bfeceae466549bfd9635e084a9aae6d9c8e1300b1c645ff1ef7b702ebb
 size 5150404397
diff --git a/model-00039-of-00047.safetensors b/model-00039-of-00047.safetensors
index 4eeac46ef1c976dee2b961de06b3c2239b9ca43c..fe365bbce0beaf6cfd406e7153c82b52c6a9d041 100644
--- a/model-00039-of-00047.safetensors
+++ b/model-00039-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1b37ae3298d03d9a44789a9dea69f2ea92955913ef371c4cde5f368eb02ecb78
+oid sha256:9b0f091b53e8a19169eb3dadf3eacd379c5ecd1233af2df9a2593a611a8285ca
 size 5286386501
diff --git a/model-00040-of-00047.safetensors b/model-00040-of-00047.safetensors
index f3e78fa73034c84af7b585a4d384775b54506b49..3bd64bd158086116ec72e259ef86632f92bd5fea 100644
--- a/model-00040-of-00047.safetensors
+++ b/model-00040-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bbd9b69cfbe060690d965039b9786e4b2992ae91b58ae56ab1a858abe7aa164c
+oid sha256:93b9ef8e99d2df0adea7a999ea5a81b6f156637eb949bd9d93423cdf2c125448
 size 5267641930
diff --git a/model-00041-of-00047.safetensors b/model-00041-of-00047.safetensors
index 364d61bc2c325832fa3849efcd3dcd10fe1db682..a3e81cb854ef07b7193452397544c15058967733 100644
--- a/model-00041-of-00047.safetensors
+++ b/model-00041-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:858cd465af14715644d2d92ccdc493d1af1a2d3298637cbae0b5a12f55ce619f
+oid sha256:fbebcd01a2bd2880f37d10622d7772e9918534033bf7d130815ee9bdc483cf42
 size 5169148832
diff --git a/model-00042-of-00047.safetensors b/model-00042-of-00047.safetensors
index fa25307d04b59e794d529c0d83bd3cbf724cbfec..01a2a51e1ccc50ef5fc6a731966ca79e57c1df70 100644
--- a/model-00042-of-00047.safetensors
+++ b/model-00042-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2a5e8cbc925884be9278cc8fc66da5856c2fc82799ea411d0b1c4c5b2160971f
+oid sha256:f3100fe613ad5f6d5b336bc10130d91684d52cbe5fa17eb3b02c0793b7cd2ce5
 size 5267642050
diff --git a/model-00043-of-00047.safetensors b/model-00043-of-00047.safetensors
index aff116b4df435cd42823df2ff46297e0f1781e8b..64b5abbd6a5a8131d77168b446201217c895e6d3 100644
--- a/model-00043-of-00047.safetensors
+++ b/model-00043-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1c7ccc776c95446925e0cfd27917438642ea05e1cef36f91a571eef3f01bc2ed
+oid sha256:4f9a413034dd2d34d9b6e06f638a4ff982b82be693c1701e78452057923a2445
 size 5286386411
diff --git a/model-00044-of-00047.safetensors b/model-00044-of-00047.safetensors
index 14c642a5413329507c05d02bccec3f50e359a754..e09598079528e1c995f157cf1961a51613fbb0e7 100644
--- a/model-00044-of-00047.safetensors
+++ b/model-00044-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:66cc59c2647dfa6db59accc5452db6461fff8eb4fc2bf952a92fbdc3b56af501
+oid sha256:b920b7c3b6da779a1966a4a7a3f268842b0eac1995d783cbc3df51f1027fe897
 size 5150404411
diff --git a/model-00045-of-00047.safetensors b/model-00045-of-00047.safetensors
index 0ab4212184c050b92f61a9de5486cdf94dbedd19..f7a7ba66ef89cdd1c0c1e5b5aa1e599ab224e937 100644
--- a/model-00045-of-00047.safetensors
+++ b/model-00045-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9999b0f020323454efecb565c7cd711a4f750257a7280498ea55aaa005b077f4
+oid sha256:050567bfb45ef6db5acab0ddcf9366604ffce2109d048d50a048cbf1aeac3cc1
 size 5267642038
diff --git a/model-00046-of-00047.safetensors b/model-00046-of-00047.safetensors
index ae7ff2c175437a46156bdf81e914303a74c3ac77..308b2982ab4aee72551a95d2f4f5ec623b992e8b 100644
--- a/model-00046-of-00047.safetensors
+++ b/model-00046-of-00047.safetensors
@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:05a1513a7f548705634a59d3f523cf16d6d866d3d7fa3ae71922e8701678f41e
-size 4482728585
+oid sha256:91da2f99bf45232f376d5a32c27c6542a56f234f4dc95d4fe10920d5336d9d0e
+size 2810846901
diff --git a/model-00047-of-00047.safetensors b/model-00047-of-00047.safetensors
deleted file mode 100644
index 3b376de87ca32432dac26131b2d0a734b344a13a..0000000000000000000000000000000000000000
--- a/model-00047-of-00047.safetensors
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:3643a0fba642df38f19376c9744b185543e834c3434aa8bc1c84feb038d44371
-size 2842991399
diff --git a/model.safetensors.index.json b/model.safetensors.index.json
index 31b541ac9cb50e35bbc6149857588ddcbe4936a0..42ed8c8fb996c92db80e83cd685e9037764c5d47 100644
--- a/model.safetensors.index.json
+++ b/model.safetensors.index.json
@@ -1,7 +1,7 @@
 {
   "metadata": {
-    "total_size": 246915401252,
-    "total_parameters": 743377019904
+    "total_size": 242400528210,
+    "total_parameters": 753329940480
   },
   "weight_map": {
     "lm_head.biases": "model-00046-of-00047.safetensors",
@@ -3359,107 +3359,56 @@
     "model.layers.9.self_attn.unembed_out.scales": "model-00004-of-00047.safetensors",
     "model.layers.9.self_attn.unembed_out.weight": "model-00004-of-00047.safetensors",
     "model.norm.weight": "model-00046-of-00047.safetensors",
-    "mtp.eh_proj.weight": "model-00046-of-00047.safetensors",
-    "mtp.enorm.weight": "model-00046-of-00047.safetensors",
-    "mtp.hnorm.weight": "model-00046-of-00047.safetensors",
-    "mtp.layer.input_layernorm.weight": "model-00047-of-00047.safetensors",
-    "mtp.layer.mlp.gate.e_score_correction_bias": "model-00047-of-00047.safetensors",
-    "mtp.layer.mlp.gate.weight": "model-00047-of-00047.safetensors",
-    "mtp.layer.mlp.shared_experts.down_proj.biases": "model-00047-of-00047.safetensors",
-    "mtp.layer.mlp.shared_experts.down_proj.scales": "model-00047-of-00047.safetensors",
-    "mtp.layer.mlp.shared_experts.down_proj.weight": "model-00047-of-00047.safetensors",
-    "mtp.layer.mlp.shared_experts.gate_proj.biases": "model-00047-of-00047.safetensors",
-    "mtp.layer.mlp.shared_experts.gate_proj.scales": "model-00047-of-00047.safetensors",
-    "mtp.layer.mlp.shared_experts.gate_proj.weight": "model-00047-of-00047.safetensors",
-    "mtp.layer.mlp.shared_experts.up_proj.biases": "model-00047-of-00047.safetensors",
-    "mtp.layer.mlp.shared_experts.up_proj.scales": "model-00047-of-00047.safetensors",
-    "mtp.layer.mlp.shared_experts.up_proj.weight": "model-00047-of-00047.safetensors",
-    "mtp.layer.mlp.switch_mlp.down_proj.biases": "model-00047-of-00047.safetensors",
-    "mtp.layer.mlp.switch_mlp.down_proj.scales": "model-00047-of-00047.safetensors",
-    "mtp.layer.mlp.switch_mlp.down_proj.weight": "model-00047-of-00047.safetensors",
-    "mtp.layer.mlp.switch_mlp.gate_proj.biases": "model-00046-of-00047.safetensors",
-    "mtp.layer.mlp.switch_mlp.gate_proj.scales": "model-00046-of-00047.safetensors",
-    "mtp.layer.mlp.switch_mlp.gate_proj.weight": "model-00046-of-00047.safetensors",
-    "mtp.layer.mlp.switch_mlp.up_proj.biases": "model-00047-of-00047.safetensors",
-    "mtp.layer.mlp.switch_mlp.up_proj.scales": "model-00047-of-00047.safetensors",
-    "mtp.layer.mlp.switch_mlp.up_proj.weight": "model-00047-of-00047.safetensors",
-    "mtp.layer.post_attention_layernorm.weight": "model-00047-of-00047.safetensors",
-    "mtp.layer.self_attn.embed_q.biases": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.embed_q.scales": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.embed_q.weight": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.indexer.k_norm.bias": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.indexer.k_norm.weight": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.indexer.weights_proj.weight": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.indexer.wk.weight": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.indexer.wq_b.weight": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.kv_a_layernorm.weight": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.kv_a_proj_with_mqa.biases": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.kv_a_proj_with_mqa.scales": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.kv_a_proj_with_mqa.weight": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.o_proj.biases": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.o_proj.scales": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.o_proj.weight": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.q_a_layernorm.weight": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.q_a_proj.biases": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.q_a_proj.scales": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.q_a_proj.weight": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.q_b_proj.biases": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.q_b_proj.scales": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.q_b_proj.weight": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.unembed_out.biases": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.unembed_out.scales": "model-00046-of-00047.safetensors",
-    "mtp.layer.self_attn.unembed_out.weight": "model-00046-of-00047.safetensors",
-    "mtp.shared_head.weight": "model-00047-of-00047.safetensors",
-    "model.layers.78.shared_head.norm.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.unembed_out.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.unembed_out.scales": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.unembed_out.biases": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.q_b_proj.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.q_b_proj.biases": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.q_a_proj.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.q_a_proj.biases": "model-mtp-layer78.safetensors",
-    "model.layers.78.mlp.switch_mlp.gate_proj.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.o_proj.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.kv_a_proj_with_mqa.scales": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.kv_a_proj_with_mqa.biases": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.kv_a_layernorm.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.q_b_proj.scales": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.q_a_layernorm.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.o_proj.scales": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.indexer.wq_b.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.mlp.switch_mlp.up_proj.biases": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.indexer.wk.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.embed_q.biases": "model-mtp-layer78.safetensors",
-    "model.layers.78.post_attention_layernorm.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.mlp.switch_mlp.up_proj.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.o_proj.biases": "model-mtp-layer78.safetensors",
-    "model.layers.78.mlp.shared_experts.gate_proj.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.mlp.switch_mlp.down_proj.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.eh_proj.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.enorm.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.hnorm.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.input_layernorm.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.mlp.gate.e_score_correction_bias": "model-mtp-layer78.safetensors",
     "model.layers.78.mlp.gate.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.mlp.switch_mlp.gate_proj.scales": "model-mtp-layer78.safetensors",
+    "model.layers.78.mlp.shared_experts.down_proj.biases": "model-mtp-layer78.safetensors",
+    "model.layers.78.mlp.shared_experts.down_proj.scales": "model-mtp-layer78.safetensors",
+    "model.layers.78.mlp.shared_experts.down_proj.weight": "model-mtp-layer78.safetensors",
     "model.layers.78.mlp.shared_experts.gate_proj.biases": "model-mtp-layer78.safetensors",
-    "model.layers.78.eh_proj.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.mlp.shared_experts.gate_proj.scales": "model-mtp-layer78.safetensors",
+    "model.layers.78.mlp.shared_experts.gate_proj.weight": "model-mtp-layer78.safetensors",
     "model.layers.78.mlp.shared_experts.up_proj.biases": "model-mtp-layer78.safetensors",
-    "model.layers.78.mlp.switch_mlp.down_proj.scales": "model-mtp-layer78.safetensors",
-    "model.layers.78.enorm.weight": "model-mtp-layer78.safetensors",
     "model.layers.78.mlp.shared_experts.up_proj.scales": "model-mtp-layer78.safetensors",
-    "model.layers.78.mlp.shared_experts.down_proj.biases": "model-mtp-layer78.safetensors",
+    "model.layers.78.mlp.shared_experts.up_proj.weight": "model-mtp-layer78.safetensors",
     "model.layers.78.mlp.switch_mlp.down_proj.biases": "model-mtp-layer78.safetensors",
+    "model.layers.78.mlp.switch_mlp.down_proj.scales": "model-mtp-layer78.safetensors",
+    "model.layers.78.mlp.switch_mlp.down_proj.weight": "model-mtp-layer78.safetensors",
     "model.layers.78.mlp.switch_mlp.gate_proj.biases": "model-mtp-layer78.safetensors",
+    "model.layers.78.mlp.switch_mlp.gate_proj.scales": "model-mtp-layer78.safetensors",
+    "model.layers.78.mlp.switch_mlp.gate_proj.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.mlp.switch_mlp.up_proj.biases": "model-mtp-layer78.safetensors",
     "model.layers.78.mlp.switch_mlp.up_proj.scales": "model-mtp-layer78.safetensors",
-    "model.layers.78.mlp.shared_experts.gate_proj.scales": "model-mtp-layer78.safetensors",
-    "model.layers.78.mlp.shared_experts.down_proj.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.mlp.shared_experts.up_proj.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.mlp.switch_mlp.up_proj.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.post_attention_layernorm.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.embed_q.biases": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.embed_q.scales": "model-mtp-layer78.safetensors",
     "model.layers.78.self_attn.embed_q.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.hnorm.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.mlp.gate.e_score_correction_bias": "model-mtp-layer78.safetensors",
     "model.layers.78.self_attn.indexer.k_norm.bias": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.q_a_proj.scales": "model-mtp-layer78.safetensors",
-    "model.layers.78.mlp.shared_experts.down_proj.scales": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.embed_q.scales": "model-mtp-layer78.safetensors",
-    "model.layers.78.input_layernorm.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.indexer.weights_proj.weight": "model-mtp-layer78.safetensors",
     "model.layers.78.self_attn.indexer.k_norm.weight": "model-mtp-layer78.safetensors",
-    "model.layers.78.self_attn.kv_a_proj_with_mqa.weight": "model-mtp-layer78.safetensors"
+    "model.layers.78.self_attn.indexer.weights_proj.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.indexer.wk.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.indexer.wq_b.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.kv_a_layernorm.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.kv_a_proj_with_mqa.biases": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.kv_a_proj_with_mqa.scales": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.kv_a_proj_with_mqa.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.o_proj.biases": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.o_proj.scales": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.o_proj.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.q_a_layernorm.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.q_a_proj.biases": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.q_a_proj.scales": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.q_a_proj.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.q_b_proj.biases": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.q_b_proj.scales": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.q_b_proj.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.unembed_out.biases": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.unembed_out.scales": "model-mtp-layer78.safetensors",
+    "model.layers.78.self_attn.unembed_out.weight": "model-mtp-layer78.safetensors",
+    "model.layers.78.shared_head.norm.weight": "model-mtp-layer78.safetensors"
   }
 }
\ No newline at end of file