diff --git "a/debug.log" "b/debug.log"
--- "a/debug.log"
+++ "b/debug.log"
@@ -1,7 +1,7 @@
-
Loading dataset from disk:   0%|                                                              | 0/205 [00:00<?, ?it/s]
Loading dataset from disk: 100%|████████████████████████████████████████████████| 205/205 [00:00<00:00, 323853.98it/s]
-
Loading weights:   0%|                                                                        | 0/338 [00:00<?, ?it/s]
Loading weights:   0%|              | 1/338 [00:00<00:00, 14926.35it/s, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|               | 1/338 [00:00<00:00, 6898.53it/s, Materializing param=model.embed_tokens.weight]
Loading weights:   1%|   | 2/338 [00:00<00:00, 6523.02it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|   | 2/338 [00:00<00:00, 5391.14it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|     | 3/338 [00:00<00:00, 6014.78it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   1%|     | 3/338 [00:00<00:00, 5186.69it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   1%|     | 4/338 [00:00<00:00, 5805.27it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|     | 4/338 [00:00<00:00, 5361.85it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|       | 5/338 [00:00<00:00, 5838.40it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   1%|       | 5/338 [00:00<00:00, 5497.12it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   2%| | 6/338 [00:00<00:00, 5983.32it/s, Materializing param=model.layers.0.post_attention_layernorm.
Loading weights:   2%| | 6/338 [00:00<00:00, 5682.06it/s, Materializing param=model.layers.0.post_attention_layernorm.
Loading weights:   2%|    | 7/338 [00:00<00:00, 6025.06it/s, Materializing param=model.layers.0.self_attn.k_proj.bias]
Loading weights:   2%|    | 7/338 [00:00<00:00, 5762.54it/s, Materializing param=model.layers.0.self_attn.k_proj.bias]
Loading weights:   2%|  | 8/338 [00:00<00:00, 6100.81it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|  | 8/338 [00:00<00:00, 5861.04it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   3%|  | 9/338 [00:00<00:00, 6098.34it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   3%|  | 9/338 [00:00<00:00, 5885.37it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   3%|   | 10/338 [00:00<00:00, 6172.63it/s, Materializing param=model.layers.0.self_attn.q_proj.bias]
Loading weights:   3%|   | 10/338 [00:00<00:00, 5975.64it/s, Materializing param=model.layers.0.self_attn.q_proj.bias]
Loading weights:   3%| | 11/338 [00:00<00:00, 6183.80it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%| | 11/338 [00:00<00:00, 6002.78it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   4%|   | 12/338 [00:00<00:00, 6248.50it/s, Materializing param=model.layers.0.self_attn.v_proj.bias]
Loading weights:   4%|   | 12/338 [00:00<00:00, 6079.44it/s, Materializing param=model.layers.0.self_attn.v_proj.bias]
Loading weights:   4%| | 13/338 [00:00<00:00, 6031.63it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   4%| | 13/338 [00:00<00:00, 5887.70it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   4%|  | 14/338 [00:00<00:00, 6064.26it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   4%|  | 14/338 [00:00<00:00, 5930.14it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   4%|▏   | 15/338 [00:00<00:00, 6112.96it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   4%|▏   | 15/338 [00:00<00:00, 5967.99it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   5%|▏   | 16/338 [00:00<00:00, 5954.12it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   5%|▏   | 16/338 [00:00<00:00, 5648.42it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   5%|▎     | 17/338 [00:00<00:00, 4296.15it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   5%|▎     | 17/338 [00:00<00:00, 4236.67it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   5%| | 18/338 [00:00<00:00, 4386.58it/s, Materializing param=model.layers.1.post_attention_layernorm
Loading weights:   5%| | 18/338 [00:00<00:00, 4329.23it/s, Materializing param=model.layers.1.post_attention_layernorm
Loading weights:   6%|▏  | 19/338 [00:00<00:00, 4444.85it/s, Materializing param=model.layers.1.self_attn.k_proj.bias]
Loading weights:   6%|▏  | 19/338 [00:00<00:00, 4388.80it/s, Materializing param=model.layers.1.self_attn.k_proj.bias]
Loading weights:   6%| | 20/338 [00:00<00:00, 4525.09it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   6%| | 20/338 [00:00<00:00, 4473.69it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   6%| | 21/338 [00:00<00:00, 4494.13it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   6%| | 21/338 [00:00<00:00, 4443.34it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   7%|▏  | 22/338 [00:00<00:00, 4500.33it/s, Materializing param=model.layers.1.self_attn.q_proj.bias]
Loading weights:   7%|▏  | 22/338 [00:00<00:00, 4451.48it/s, Materializing param=model.layers.1.self_attn.q_proj.bias]
Loading weights:   7%| | 23/338 [00:00<00:00, 4570.69it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   7%| | 23/338 [00:00<00:00, 4525.02it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   7%|▏  | 24/338 [00:00<00:00, 4644.64it/s, Materializing param=model.layers.1.self_attn.v_proj.bias]
Loading weights:   7%|▏  | 24/338 [00:00<00:00, 4545.85it/s, Materializing param=model.layers.1.self_attn.v_proj.bias]
Loading weights:   7%| | 25/338 [00:00<00:00, 4633.57it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   7%| | 25/338 [00:00<00:00, 4590.36it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   8%|▏ | 26/338 [00:00<00:00, 4687.58it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   8%|▏ | 26/338 [00:00<00:00, 4644.85it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   8%|▎   | 27/338 [00:00<00:00, 4747.67it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   8%|▎   | 27/338 [00:00<00:00, 4705.46it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   8%|▎   | 28/338 [00:00<00:00, 4793.69it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   8%|▎   | 28/338 [00:00<00:00, 4751.60it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   9%|▌     | 29/338 [00:00<00:00, 4847.94it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   9%|▌     | 29/338 [00:00<00:00, 4806.18it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   9%| | 30/338 [00:00<00:00, 4616.90it/s, Materializing param=model.layers.2.post_attention_layernorm
Loading weights:   9%| | 30/338 [00:00<00:00, 4575.77it/s, Materializing param=model.layers.2.post_attention_layernorm
Loading weights:   9%|▎  | 31/338 [00:00<00:00, 4664.52it/s, Materializing param=model.layers.2.self_attn.k_proj.bias]
Loading weights:   9%|▎  | 31/338 [00:00<00:00, 4627.83it/s, Materializing param=model.layers.2.self_attn.k_proj.bias]
Loading weights:   9%| | 32/338 [00:00<00:00, 4713.36it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   9%| | 32/338 [00:00<00:00, 4677.88it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:  10%| | 33/338 [00:00<00:00, 4619.43it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:  10%| | 33/338 [00:00<00:00, 4584.09it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:  10%|▎  | 34/338 [00:00<00:00, 4656.08it/s, Materializing param=model.layers.2.self_attn.q_proj.bias]
Loading weights:  10%|▎  | 34/338 [00:00<00:00, 4623.92it/s, Materializing param=model.layers.2.self_attn.q_proj.bias]
Loading weights:  10%| | 35/338 [00:00<00:00, 4702.44it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:  10%| | 35/338 [00:00<00:00, 4670.27it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:  11%|▎  | 36/338 [00:00<00:00, 4741.26it/s, Materializing param=model.layers.2.self_attn.v_proj.bias]
Loading weights:  11%|▎  | 36/338 [00:00<00:00, 4708.44it/s, Materializing param=model.layers.2.self_attn.v_proj.bias]
Loading weights:  11%| | 37/338 [00:00<00:00, 4783.74it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:  11%| | 37/338 [00:00<00:00, 4753.70it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:  11%|▏ | 38/338 [00:00<00:00, 4819.29it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:  11%|▏ | 38/338 [00:00<00:00, 4787.16it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:  12%|▍   | 39/338 [00:00<00:00, 4859.28it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:  12%|▍   | 39/338 [00:00<00:00, 4828.30it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:  12%|▍   | 40/338 [00:00<00:00, 4824.92it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:  12%|▍   | 40/338 [00:00<00:00, 4794.31it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:  12%|▋     | 41/338 [00:00<00:00, 4861.93it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:  12%|▋     | 41/338 [00:00<00:00, 4831.74it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:  12%| | 42/338 [00:00<00:00, 4803.42it/s, Materializing param=model.layers.3.post_attention_layernorm
Loading weights:  12%| | 42/338 [00:00<00:00, 4772.97it/s, Materializing param=model.layers.3.post_attention_layernorm
Loading weights:  13%|▍  | 43/338 [00:00<00:00, 4838.37it/s, Materializing param=model.layers.3.self_attn.k_proj.bias]
Loading weights:  13%|▍  | 43/338 [00:00<00:00, 4809.47it/s, Materializing param=model.layers.3.self_attn.k_proj.bias]
Loading weights:  13%|▏| 44/338 [00:00<00:00, 4873.62it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  13%|▏| 44/338 [00:00<00:00, 4845.85it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  13%|▏| 45/338 [00:00<00:00, 4851.90it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  13%|▏| 45/338 [00:00<00:00, 4822.89it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  14%|▍  | 46/338 [00:00<00:00, 4884.26it/s, Materializing param=model.layers.3.self_attn.q_proj.bias]
Loading weights:  14%|▍  | 46/338 [00:00<00:00, 4856.96it/s, Materializing param=model.layers.3.self_attn.q_proj.bias]
Loading weights:  14%|▏| 47/338 [00:00<00:00, 4917.73it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  14%|▏| 47/338 [00:00<00:00, 4890.77it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  14%|▍  | 48/338 [00:00<00:00, 4935.08it/s, Materializing param=model.layers.3.self_attn.v_proj.bias]
Loading weights:  14%|▍  | 48/338 [00:00<00:00, 4908.49it/s, Materializing param=model.layers.3.self_attn.v_proj.bias]
Loading weights:  14%|▏| 49/338 [00:00<00:00, 4959.36it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  14%|▏| 49/338 [00:00<00:00, 4933.29it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  15%|▎ | 50/338 [00:00<00:00, 4900.69it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  15%|▎ | 50/338 [00:00<00:00, 4873.70it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  15%|▌   | 51/338 [00:00<00:00, 4927.54it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  15%|▌   | 51/338 [00:00<00:00, 4902.02it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  15%|▌   | 52/338 [00:00<00:00, 4945.10it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  15%|▌   | 52/338 [00:00<00:00, 4920.78it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  16%|▉     | 53/338 [00:00<00:00, 4974.56it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  16%|▉     | 53/338 [00:00<00:00, 4950.96it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  16%|▏| 54/338 [00:00<00:00, 4907.21it/s, Materializing param=model.layers.4.post_attention_layernorm
Loading weights:  16%|▏| 54/338 [00:00<00:00, 4882.78it/s, Materializing param=model.layers.4.post_attention_layernorm
Loading weights:  16%|▍  | 55/338 [00:00<00:00, 4927.83it/s, Materializing param=model.layers.4.self_attn.k_proj.bias]
Loading weights:  16%|▍  | 55/338 [00:00<00:00, 4904.99it/s, Materializing param=model.layers.4.self_attn.k_proj.bias]
Loading weights:  17%|▏| 56/338 [00:00<00:00, 4955.61it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  17%|▏| 56/338 [00:00<00:00, 4933.85it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  17%|▏| 57/338 [00:00<00:00, 4978.97it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  17%|▏| 57/338 [00:00<00:00, 4956.26it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  17%|▌  | 58/338 [00:00<00:00, 5005.34it/s, Materializing param=model.layers.4.self_attn.q_proj.bias]
Loading weights:  17%|▌  | 58/338 [00:00<00:00, 4983.20it/s, Materializing param=model.layers.4.self_attn.q_proj.bias]
Loading weights:  17%|▏| 59/338 [00:00<00:00, 5026.18it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  17%|▏| 59/338 [00:00<00:00, 5004.23it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  18%|▌  | 60/338 [00:00<00:00, 5052.57it/s, Materializing param=model.layers.4.self_attn.v_proj.bias]
Loading weights:  18%|▌  | 60/338 [00:00<00:00, 5031.25it/s, Materializing param=model.layers.4.self_attn.v_proj.bias]
Loading weights:  18%|▏| 61/338 [00:00<00:00, 5072.92it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  18%|▏| 61/338 [00:00<00:00, 5051.28it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  18%|▎ | 62/338 [00:00<00:00, 5098.46it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  18%|▎ | 62/338 [00:00<00:00, 5076.66it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  19%|▋   | 63/338 [00:00<00:00, 5110.36it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  19%|▋   | 63/338 [00:00<00:00, 5087.14it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  19%|▊   | 64/338 [00:00<00:00, 5131.63it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  19%|▊   | 64/338 [00:00<00:00, 5111.01it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  19%|█▏    | 65/338 [00:00<00:00, 5150.76it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  19%|█▏    | 65/338 [00:00<00:00, 5129.44it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  20%|▏| 66/338 [00:00<00:00, 5172.54it/s, Materializing param=model.layers.5.post_attention_layernorm
Loading weights:  20%|▏| 66/338 [00:00<00:00, 5150.89it/s, Materializing param=model.layers.5.post_attention_layernorm
Loading weights:  20%|▌  | 67/338 [00:00<00:00, 5189.24it/s, Materializing param=model.layers.5.self_attn.k_proj.bias]
Loading weights:  20%|▌  | 67/338 [00:00<00:00, 5167.30it/s, Materializing param=model.layers.5.self_attn.k_proj.bias]
Loading weights:  20%|▏| 68/338 [00:00<00:00, 5208.51it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  20%|▏| 68/338 [00:00<00:00, 5187.48it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  20%|▏| 69/338 [00:00<00:00, 5226.59it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  20%|▏| 69/338 [00:00<00:00, 5205.91it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  21%|▌  | 70/338 [00:00<00:00, 5247.19it/s, Materializing param=model.layers.5.self_attn.q_proj.bias]
Loading weights:  21%|▌  | 70/338 [00:00<00:00, 5226.55it/s, Materializing param=model.layers.5.self_attn.q_proj.bias]
Loading weights:  21%|▏| 71/338 [00:00<00:00, 5264.48it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  21%|▏| 71/338 [00:00<00:00, 5243.62it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  21%|▋  | 72/338 [00:00<00:00, 5283.79it/s, Materializing param=model.layers.5.self_attn.v_proj.bias]
Loading weights:  21%|▋  | 72/338 [00:00<00:00, 5264.08it/s, Materializing param=model.layers.5.self_attn.v_proj.bias]
Loading weights:  22%|▏| 73/338 [00:00<00:00, 5032.37it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  22%|▏| 73/338 [00:00<00:00, 5012.76it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  22%|▍ | 74/338 [00:00<00:00, 5051.82it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  22%|▍ | 74/338 [00:00<00:00, 5033.95it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  22%|▉   | 75/338 [00:00<00:00, 5063.79it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  22%|▉   | 75/338 [00:00<00:00, 5046.08it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  22%|▉   | 76/338 [00:00<00:00, 5079.55it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  22%|▉   | 76/338 [00:00<00:00, 5062.13it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  23%|█▎    | 77/338 [00:00<00:00, 5099.66it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  23%|█▎    | 77/338 [00:00<00:00, 5081.92it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  23%|▏| 78/338 [00:00<00:00, 5071.40it/s, Materializing param=model.layers.6.post_attention_layernorm
Loading weights:  23%|▏| 78/338 [00:00<00:00, 5053.07it/s, Materializing param=model.layers.6.post_attention_layernorm
Loading weights:  23%|▋  | 79/338 [00:00<00:00, 5084.32it/s, Materializing param=model.layers.6.self_attn.k_proj.bias]
Loading weights:  23%|▋  | 79/338 [00:00<00:00, 5066.90it/s, Materializing param=model.layers.6.self_attn.k_proj.bias]
Loading weights:  24%|▏| 80/338 [00:00<00:00, 5102.64it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  24%|▏| 80/338 [00:00<00:00, 5086.32it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  24%|▏| 81/338 [00:00<00:00, 5111.85it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  24%|▏| 81/338 [00:00<00:00, 5095.29it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  24%|▋  | 82/338 [00:00<00:00, 5125.45it/s, Materializing param=model.layers.6.self_attn.q_proj.bias]
Loading weights:  24%|▋  | 82/338 [00:00<00:00, 5108.93it/s, Materializing param=model.layers.6.self_attn.q_proj.bias]
Loading weights:  25%|▏| 83/338 [00:00<00:00, 5143.27it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  25%|▏| 83/338 [00:00<00:00, 5127.28it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  25%|▋  | 84/338 [00:00<00:00, 5157.31it/s, Materializing param=model.layers.6.self_attn.v_proj.bias]
Loading weights:  25%|▋  | 84/338 [00:00<00:00, 5075.07it/s, Materializing param=model.layers.6.self_attn.v_proj.bias]
Loading weights:  25%|▎| 85/338 [00:00<00:00, 5051.73it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  25%|▎| 85/338 [00:00<00:00, 5034.75it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  25%|▌ | 86/338 [00:00<00:00, 5067.65it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  25%|▌ | 86/338 [00:00<00:00, 5052.53it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  26%|█   | 87/338 [00:00<00:00, 5081.67it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  26%|█   | 87/338 [00:00<00:00, 5066.92it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  26%|█   | 88/338 [00:00<00:00, 5099.67it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  26%|█   | 88/338 [00:00<00:00, 5084.57it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  26%|█▌    | 89/338 [00:00<00:00, 5113.04it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  26%|█▌    | 89/338 [00:00<00:00, 5098.10it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  27%|▎| 90/338 [00:00<00:00, 5024.79it/s, Materializing param=model.layers.7.post_attention_layernorm
Loading weights:  27%|▎| 90/338 [00:00<00:00, 5007.13it/s, Materializing param=model.layers.7.post_attention_layernorm
Loading weights:  27%|▊  | 91/338 [00:00<00:00, 5039.17it/s, Materializing param=model.layers.7.self_attn.k_proj.bias]
Loading weights:  27%|▊  | 91/338 [00:00<00:00, 5024.71it/s, Materializing param=model.layers.7.self_attn.k_proj.bias]
Loading weights:  27%|▎| 92/338 [00:00<00:00, 5051.53it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  27%|▎| 92/338 [00:00<00:00, 5037.48it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  28%|▎| 93/338 [00:00<00:00, 5068.68it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  28%|▎| 93/338 [00:00<00:00, 5055.08it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  28%|▊  | 94/338 [00:00<00:00, 5082.10it/s, Materializing param=model.layers.7.self_attn.q_proj.bias]
Loading weights:  28%|▊  | 94/338 [00:00<00:00, 5068.12it/s, Materializing param=model.layers.7.self_attn.q_proj.bias]
Loading weights:  28%|▎| 95/338 [00:00<00:00, 5099.10it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  28%|▎| 95/338 [00:00<00:00, 5085.95it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  28%|▊  | 96/338 [00:00<00:00, 4964.41it/s, Materializing param=model.layers.7.self_attn.v_proj.bias]
Loading weights:  28%|▊  | 96/338 [00:00<00:00, 4949.94it/s, Materializing param=model.layers.7.self_attn.v_proj.bias]
Loading weights:  29%|▎| 97/338 [00:00<00:00, 4933.52it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  29%|▎| 97/338 [00:00<00:00, 4919.56it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  29%|▌ | 98/338 [00:00<00:00, 4948.32it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  29%|▌ | 98/338 [00:00<00:00, 4934.77it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  29%|█▏  | 99/338 [00:00<00:00, 4959.64it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  29%|█▏  | 99/338 [00:00<00:00, 4946.76it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  30%|▉  | 100/338 [00:00<00:00, 4975.86it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  30%|▉  | 100/338 [00:00<00:00, 4963.03it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  30%|█▍   | 101/338 [00:00<00:00, 4987.99it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  30%|█▍   | 101/338 [00:00<00:00, 4975.10it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  30%|▎| 102/338 [00:00<00:00, 5004.14it/s, Materializing param=model.layers.8.post_attention_layernor
Loading weights:  30%|▎| 102/338 [00:00<00:00, 4991.30it/s, Materializing param=model.layers.8.post_attention_layernor
Loading weights:  30%|▌ | 103/338 [00:00<00:00, 5015.65it/s, Materializing param=model.layers.8.self_attn.k_proj.bias]
Loading weights:  30%|▌ | 103/338 [00:00<00:00, 5002.93it/s, Materializing param=model.layers.8.self_attn.k_proj.bias]
Loading weights:  31%|▎| 104/338 [00:00<00:00, 5030.59it/s, Materializing param=model.layers.8.self_attn.k_proj.weight
Loading weights:  31%|▎| 104/338 [00:00<00:00, 5017.86it/s, Materializing param=model.layers.8.self_attn.k_proj.weight
Loading weights:  31%|▎| 105/338 [00:00<00:00, 5041.40it/s, Materializing param=model.layers.8.self_attn.o_proj.weight
Loading weights:  31%|▎| 105/338 [00:00<00:00, 5029.14it/s, Materializing param=model.layers.8.self_attn.o_proj.weight
Loading weights:  31%|▋ | 106/338 [00:00<00:00, 5055.96it/s, Materializing param=model.layers.8.self_attn.q_proj.bias]
Loading weights:  31%|▋ | 106/338 [00:00<00:00, 5043.63it/s, Materializing param=model.layers.8.self_attn.q_proj.bias]
Loading weights:  32%|▎| 107/338 [00:00<00:00, 5066.73it/s, Materializing param=model.layers.8.self_attn.q_proj.weight
Loading weights:  32%|▎| 107/338 [00:00<00:00, 5054.80it/s, Materializing param=model.layers.8.self_attn.q_proj.weight
Loading weights:  32%|▋ | 108/338 [00:00<00:00, 5081.95it/s, Materializing param=model.layers.8.self_attn.v_proj.bias]
Loading weights:  32%|▋ | 108/338 [00:00<00:00, 5069.61it/s, Materializing param=model.layers.8.self_attn.v_proj.bias]
Loading weights:  32%|▎| 109/338 [00:00<00:00, 5092.05it/s, Materializing param=model.layers.8.self_attn.v_proj.weight
Loading weights:  32%|▎| 109/338 [00:00<00:00, 5079.49it/s, Materializing param=model.layers.8.self_attn.v_proj.weight
Loading weights:  33%|▎| 110/338 [00:00<00:00, 5105.72it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  33%|▎| 110/338 [00:00<00:00, 5093.72it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  33%|▉  | 111/338 [00:00<00:00, 5116.86it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  33%|▉  | 111/338 [00:00<00:00, 5104.74it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  33%|▉  | 112/338 [00:00<00:00, 5130.59it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  33%|▉  | 112/338 [00:00<00:00, 5118.68it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  33%|█▋   | 113/338 [00:00<00:00, 5142.53it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  33%|█▋   | 113/338 [00:00<00:00, 5130.34it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  34%|▎| 114/338 [00:00<00:00, 5155.43it/s, Materializing param=model.layers.9.post_attention_layernor
Loading weights:  34%|▎| 114/338 [00:00<00:00, 5143.23it/s, Materializing param=model.layers.9.post_attention_layernor
Loading weights:  34%|▋ | 115/338 [00:00<00:00, 5103.53it/s, Materializing param=model.layers.9.self_attn.k_proj.bias]
Loading weights:  34%|▋ | 115/338 [00:00<00:00, 5090.77it/s, Materializing param=model.layers.9.self_attn.k_proj.bias]
Loading weights:  34%|▎| 116/338 [00:00<00:00, 5114.57it/s, Materializing param=model.layers.9.self_attn.k_proj.weight
Loading weights:  34%|▎| 116/338 [00:00<00:00, 5103.20it/s, Materializing param=model.layers.9.self_attn.k_proj.weight
Loading weights:  35%|▎| 117/338 [00:00<00:00, 5120.82it/s, Materializing param=model.layers.9.self_attn.o_proj.weight
Loading weights:  35%|▎| 117/338 [00:00<00:00, 5109.25it/s, Materializing param=model.layers.9.self_attn.o_proj.weight
Loading weights:  35%|▋ | 118/338 [00:00<00:00, 5133.15it/s, Materializing param=model.layers.9.self_attn.q_proj.bias]
Loading weights:  35%|▋ | 118/338 [00:00<00:00, 5122.20it/s, Materializing param=model.layers.9.self_attn.q_proj.bias]
Loading weights:  35%|▎| 119/338 [00:00<00:00, 5071.66it/s, Materializing param=model.layers.9.self_attn.q_proj.weight
Loading weights:  35%|▎| 119/338 [00:00<00:00, 5059.12it/s, Materializing param=model.layers.9.self_attn.q_proj.weight
Loading weights:  36%|▋ | 120/338 [00:00<00:00, 5082.57it/s, Materializing param=model.layers.9.self_attn.v_proj.bias]
Loading weights:  36%|▋ | 120/338 [00:00<00:00, 5071.35it/s, Materializing param=model.layers.9.self_attn.v_proj.bias]
Loading weights:  36%|▎| 121/338 [00:00<00:00, 5088.34it/s, Materializing param=model.layers.9.self_attn.v_proj.weight
Loading weights:  36%|▎| 121/338 [00:00<00:00, 5077.19it/s, Materializing param=model.layers.9.self_attn.v_proj.weight
Loading weights:  36%|▎| 122/338 [00:00<00:00, 5097.73it/s, Materializing param=model.layers.10.input_layernorm.weight
Loading weights:  36%|▎| 122/338 [00:00<00:00, 5086.43it/s, Materializing param=model.layers.10.input_layernorm.weight
Loading weights:  36%|▋ | 123/338 [00:00<00:00, 5109.38it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  36%|▋ | 123/338 [00:00<00:00, 5098.12it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  37%|▋ | 124/338 [00:00<00:00, 5118.02it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  37%|▋ | 124/338 [00:00<00:00, 5106.82it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  37%|█▍  | 125/338 [00:00<00:00, 5129.52it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  37%|█▍  | 125/338 [00:00<00:00, 5117.90it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  37%|▎| 126/338 [00:00<00:00, 5137.33it/s, Materializing param=model.layers.10.post_attention_layerno
Loading weights:  37%|▎| 126/338 [00:00<00:00, 5125.92it/s, Materializing param=model.layers.10.post_attention_layerno
Loading weights:  38%|▍| 127/338 [00:00<00:00, 5147.48it/s, Materializing param=model.layers.10.self_attn.k_proj.bias]
Loading weights:  38%|▍| 127/338 [00:00<00:00, 5136.46it/s, Materializing param=model.layers.10.self_attn.k_proj.bias]
Loading weights:  38%|▍| 128/338 [00:00<00:00, 5156.02it/s, Materializing param=model.layers.10.self_attn.k_proj.weigh
Loading weights:  38%|▍| 128/338 [00:00<00:00, 5145.00it/s, Materializing param=model.layers.10.self_attn.k_proj.weigh
Loading weights:  38%|▍| 129/338 [00:00<00:00, 5095.11it/s, Materializing param=model.layers.10.self_attn.o_proj.weigh
Loading weights:  38%|▍| 129/338 [00:00<00:00, 5083.86it/s, Materializing param=model.layers.10.self_attn.o_proj.weigh
Loading weights:  38%|▍| 130/338 [00:00<00:00, 5105.47it/s, Materializing param=model.layers.10.self_attn.q_proj.bias]
Loading weights:  38%|▍| 130/338 [00:00<00:00, 5094.98it/s, Materializing param=model.layers.10.self_attn.q_proj.bias]
Loading weights:  39%|▍| 131/338 [00:00<00:00, 5113.58it/s, Materializing param=model.layers.10.self_attn.q_proj.weigh
Loading weights:  39%|▍| 131/338 [00:00<00:00, 5103.03it/s, Materializing param=model.layers.10.self_attn.q_proj.weigh
Loading weights:  39%|▍| 132/338 [00:00<00:00, 5124.85it/s, Materializing param=model.layers.10.self_attn.v_proj.bias]
Loading weights:  39%|▍| 132/338 [00:00<00:00, 5114.49it/s, Materializing param=model.layers.10.self_attn.v_proj.bias]
Loading weights:  39%|▍| 133/338 [00:00<00:00, 5133.27it/s, Materializing param=model.layers.10.self_attn.v_proj.weigh
Loading weights:  39%|▍| 133/338 [00:00<00:00, 5122.66it/s, Materializing param=model.layers.10.self_attn.v_proj.weigh
Loading weights:  40%|▍| 134/338 [00:00<00:00, 5144.31it/s, Materializing param=model.layers.11.input_layernorm.weight
Loading weights:  40%|▍| 134/338 [00:00<00:00, 5133.97it/s, Materializing param=model.layers.11.input_layernorm.weight
Loading weights:  40%|▊ | 135/338 [00:00<00:00, 5126.91it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  40%|▊ | 135/338 [00:00<00:00, 5115.98it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  40%|▊ | 136/338 [00:00<00:00, 5027.55it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  40%|▊ | 136/338 [00:00<00:00, 5017.24it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  41%|█▌  | 137/338 [00:00<00:00, 5037.96it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  41%|█▌  | 137/338 [00:00<00:00, 5027.87it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  41%|▍| 138/338 [00:00<00:00, 5046.15it/s, Materializing param=model.layers.11.post_attention_layerno
Loading weights:  41%|▍| 138/338 [00:00<00:00, 5036.23it/s, Materializing param=model.layers.11.post_attention_layerno
Loading weights:  41%|▍| 139/338 [00:00<00:00, 5056.88it/s, Materializing param=model.layers.11.self_attn.k_proj.bias]
Loading weights:  41%|▍| 139/338 [00:00<00:00, 5047.78it/s, Materializing param=model.layers.11.self_attn.k_proj.bias]
Loading weights:  41%|▍| 140/338 [00:00<00:00, 5065.93it/s, Materializing param=model.layers.11.self_attn.k_proj.weigh
Loading weights:  41%|▍| 140/338 [00:00<00:00, 5056.47it/s, Materializing param=model.layers.11.self_attn.k_proj.weigh
Loading weights:  42%|▍| 141/338 [00:00<00:00, 5077.07it/s, Materializing param=model.layers.11.self_attn.o_proj.weigh
Loading weights:  42%|▍| 141/338 [00:00<00:00, 5067.54it/s, Materializing param=model.layers.11.self_attn.o_proj.weigh
Loading weights:  42%|▍| 142/338 [00:00<00:00, 5085.13it/s, Materializing param=model.layers.11.self_attn.q_proj.bias]
Loading weights:  42%|▍| 142/338 [00:00<00:00, 5075.69it/s, Materializing param=model.layers.11.self_attn.q_proj.bias]
Loading weights:  42%|▍| 143/338 [00:00<00:00, 5096.01it/s, Materializing param=model.layers.11.self_attn.q_proj.weigh
Loading weights:  42%|▍| 143/338 [00:00<00:00, 5086.72it/s, Materializing param=model.layers.11.self_attn.q_proj.weigh
Loading weights:  43%|▍| 144/338 [00:00<00:00, 5104.46it/s, Materializing param=model.layers.11.self_attn.v_proj.bias]
Loading weights:  43%|▍| 144/338 [00:00<00:00, 5094.81it/s, Materializing param=model.layers.11.self_attn.v_proj.bias]
Loading weights:  43%|▍| 145/338 [00:00<00:00, 5114.36it/s, Materializing param=model.layers.11.self_attn.v_proj.weigh
Loading weights:  43%|▍| 145/338 [00:00<00:00, 5105.13it/s, Materializing param=model.layers.11.self_attn.v_proj.weigh
Loading weights:  43%|▍| 146/338 [00:00<00:00, 5122.92it/s, Materializing param=model.layers.12.input_layernorm.weight
Loading weights:  43%|▍| 146/338 [00:00<00:00, 5113.47it/s, Materializing param=model.layers.12.input_layernorm.weight
Loading weights:  43%|▊ | 147/338 [00:00<00:00, 5132.68it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  43%|▊ | 147/338 [00:00<00:00, 5123.42it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  44%|▉ | 148/338 [00:00<00:00, 5017.52it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  44%|▉ | 148/338 [00:00<00:00, 5008.16it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  44%|█▊  | 149/338 [00:00<00:00, 5027.12it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  44%|█▊  | 149/338 [00:00<00:00, 5018.32it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  44%|▍| 150/338 [00:00<00:00, 5035.34it/s, Materializing param=model.layers.12.post_attention_layerno
Loading weights:  44%|▍| 150/338 [00:00<00:00, 5026.49it/s, Materializing param=model.layers.12.post_attention_layerno
Loading weights:  45%|▍| 151/338 [00:00<00:00, 5045.65it/s, Materializing param=model.layers.12.self_attn.k_proj.bias]
Loading weights:  45%|▍| 151/338 [00:00<00:00, 5037.30it/s, Materializing param=model.layers.12.self_attn.k_proj.bias]
Loading weights:  45%|▍| 152/338 [00:00<00:00, 5030.65it/s, Materializing param=model.layers.12.self_attn.k_proj.weigh
Loading weights:  45%|▍| 152/338 [00:00<00:00, 5021.65it/s, Materializing param=model.layers.12.self_attn.k_proj.weigh
Loading weights:  45%|▍| 153/338 [00:00<00:00, 5040.24it/s, Materializing param=model.layers.12.self_attn.o_proj.weigh
Loading weights:  45%|▍| 153/338 [00:00<00:00, 5031.98it/s, Materializing param=model.layers.12.self_attn.o_proj.weigh
Loading weights:  46%|▍| 154/338 [00:00<00:00, 5045.05it/s, Materializing param=model.layers.12.self_attn.q_proj.bias]
Loading weights:  46%|▍| 154/338 [00:00<00:00, 5036.44it/s, Materializing param=model.layers.12.self_attn.q_proj.bias]
Loading weights:  46%|▍| 155/338 [00:00<00:00, 5052.51it/s, Materializing param=model.layers.12.self_attn.q_proj.weigh
Loading weights:  46%|▍| 155/338 [00:00<00:00, 5044.28it/s, Materializing param=model.layers.12.self_attn.q_proj.weigh
Loading weights:  46%|▍| 156/338 [00:00<00:00, 5062.49it/s, Materializing param=model.layers.12.self_attn.v_proj.bias]
Loading weights:  46%|▍| 156/338 [00:00<00:00, 5054.16it/s, Materializing param=model.layers.12.self_attn.v_proj.bias]
Loading weights:  46%|▍| 157/338 [00:00<00:00, 5070.58it/s, Materializing param=model.layers.12.self_attn.v_proj.weigh
Loading weights:  46%|▍| 157/338 [00:00<00:00, 5062.27it/s, Materializing param=model.layers.12.self_attn.v_proj.weigh
Loading weights:  47%|▍| 158/338 [00:00<00:00, 5080.42it/s, Materializing param=model.layers.13.input_layernorm.weight
Loading weights:  47%|▍| 158/338 [00:00<00:00, 5072.21it/s, Materializing param=model.layers.13.input_layernorm.weight
Loading weights:  47%|▉ | 159/338 [00:00<00:00, 5088.27it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  47%|▉ | 159/338 [00:00<00:00, 5079.75it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  47%|▉ | 160/338 [00:00<00:00, 5097.56it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  47%|▉ | 160/338 [00:00<00:00, 5089.29it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  48%|█▉  | 161/338 [00:00<00:00, 5104.91it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  48%|█▉  | 161/338 [00:00<00:00, 5096.55it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  48%|▍| 162/338 [00:00<00:00, 5114.27it/s, Materializing param=model.layers.13.post_attention_layerno
Loading weights:  48%|▍| 162/338 [00:00<00:00, 5105.70it/s, Materializing param=model.layers.13.post_attention_layerno
Loading weights:  48%|▍| 163/338 [00:00<00:00, 5063.33it/s, Materializing param=model.layers.13.self_attn.k_proj.bias]
Loading weights:  48%|▍| 163/338 [00:00<00:00, 5054.31it/s, Materializing param=model.layers.13.self_attn.k_proj.bias]
Loading weights:  49%|▍| 164/338 [00:00<00:00, 5071.00it/s, Materializing param=model.layers.13.self_attn.k_proj.weigh
Loading weights:  49%|▍| 164/338 [00:00<00:00, 5062.79it/s, Materializing param=model.layers.13.self_attn.k_proj.weigh
Loading weights:  49%|▍| 165/338 [00:00<00:00, 5077.96it/s, Materializing param=model.layers.13.self_attn.o_proj.weigh
Loading weights:  49%|▍| 165/338 [00:00<00:00, 5069.48it/s, Materializing param=model.layers.13.self_attn.o_proj.weigh
Loading weights:  49%|▍| 166/338 [00:00<00:00, 5086.34it/s, Materializing param=model.layers.13.self_attn.q_proj.bias]
Loading weights:  49%|▍| 166/338 [00:00<00:00, 5078.48it/s, Materializing param=model.layers.13.self_attn.q_proj.bias]
Loading weights:  49%|▍| 167/338 [00:00<00:00, 4964.38it/s, Materializing param=model.layers.13.self_attn.q_proj.weigh
Loading weights:  49%|▍| 167/338 [00:00<00:00, 4956.12it/s, Materializing param=model.layers.13.self_attn.q_proj.weigh
Loading weights:  50%|▍| 168/338 [00:00<00:00, 4946.29it/s, Materializing param=model.layers.13.self_attn.v_proj.bias]
Loading weights:  50%|▍| 168/338 [00:00<00:00, 4938.24it/s, Materializing param=model.layers.13.self_attn.v_proj.bias]
Loading weights:  50%|▌| 169/338 [00:00<00:00, 4954.55it/s, Materializing param=model.layers.13.self_attn.v_proj.weigh
Loading weights:  50%|▌| 169/338 [00:00<00:00, 4946.98it/s, Materializing param=model.layers.13.self_attn.v_proj.weigh
Loading weights:  50%|▌| 170/338 [00:00<00:00, 4913.12it/s, Materializing param=model.layers.14.input_layernorm.weight
Loading weights:  50%|▌| 170/338 [00:00<00:00, 4904.94it/s, Materializing param=model.layers.14.input_layernorm.weight
Loading weights:  51%|█ | 171/338 [00:00<00:00, 4918.98it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  51%|█ | 171/338 [00:00<00:00, 4911.33it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  51%|█ | 172/338 [00:00<00:00, 4927.33it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  51%|█ | 172/338 [00:00<00:00, 4920.04it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  51%|██  | 173/338 [00:00<00:00, 4875.26it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  51%|██  | 173/338 [00:00<00:00, 4867.54it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  51%|▌| 174/338 [00:00<00:00, 4881.80it/s, Materializing param=model.layers.14.post_attention_layerno
Loading weights:  51%|▌| 174/338 [00:00<00:00, 4874.46it/s, Materializing param=model.layers.14.post_attention_layerno
Loading weights:  52%|▌| 175/338 [00:00<00:00, 4890.26it/s, Materializing param=model.layers.14.self_attn.k_proj.bias]
Loading weights:  52%|▌| 175/338 [00:00<00:00, 4882.94it/s, Materializing param=model.layers.14.self_attn.k_proj.bias]
Loading weights:  52%|▌| 176/338 [00:00<00:00, 4864.60it/s, Materializing param=model.layers.14.self_attn.k_proj.weigh
Loading weights:  52%|▌| 176/338 [00:00<00:00, 4856.79it/s, Materializing param=model.layers.14.self_attn.k_proj.weigh
Loading weights:  52%|▌| 177/338 [00:00<00:00, 4871.85it/s, Materializing param=model.layers.14.self_attn.o_proj.weigh
Loading weights:  52%|▌| 177/338 [00:00<00:00, 4864.70it/s, Materializing param=model.layers.14.self_attn.o_proj.weigh
Loading weights:  53%|▌| 178/338 [00:00<00:00, 4880.19it/s, Materializing param=model.layers.14.self_attn.q_proj.bias]
Loading weights:  53%|▌| 178/338 [00:00<00:00, 4873.28it/s, Materializing param=model.layers.14.self_attn.q_proj.bias]
Loading weights:  53%|▌| 179/338 [00:00<00:00, 4885.00it/s, Materializing param=model.layers.14.self_attn.q_proj.weigh
Loading weights:  53%|▌| 179/338 [00:00<00:00, 4878.05it/s, Materializing param=model.layers.14.self_attn.q_proj.weigh
Loading weights:  53%|▌| 180/338 [00:00<00:00, 4892.01it/s, Materializing param=model.layers.14.self_attn.v_proj.bias]
Loading weights:  53%|▌| 180/338 [00:00<00:00, 4885.24it/s, Materializing param=model.layers.14.self_attn.v_proj.bias]
Loading weights:  54%|▌| 181/338 [00:00<00:00, 4900.93it/s, Materializing param=model.layers.14.self_attn.v_proj.weigh
Loading weights:  54%|▌| 181/338 [00:00<00:00, 4894.23it/s, Materializing param=model.layers.14.self_attn.v_proj.weigh
Loading weights:  54%|▌| 182/338 [00:00<00:00, 4895.80it/s, Materializing param=model.layers.15.input_layernorm.weight
Loading weights:  54%|▌| 182/338 [00:00<00:00, 4888.78it/s, Materializing param=model.layers.15.input_layernorm.weight
Loading weights:  54%|█ | 183/338 [00:00<00:00, 4902.27it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  54%|█ | 183/338 [00:00<00:00, 4895.11it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  54%|█ | 184/338 [00:00<00:00, 4910.24it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  54%|█ | 184/338 [00:00<00:00, 4903.34it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  55%|██▏ | 185/338 [00:00<00:00, 4917.06it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  55%|██▏ | 185/338 [00:00<00:00, 4910.03it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  55%|▌| 186/338 [00:00<00:00, 4924.88it/s, Materializing param=model.layers.15.post_attention_layerno
Loading weights:  55%|▌| 186/338 [00:00<00:00, 4917.86it/s, Materializing param=model.layers.15.post_attention_layerno
Loading weights:  55%|▌| 187/338 [00:00<00:00, 4931.22it/s, Materializing param=model.layers.15.self_attn.k_proj.bias]
Loading weights:  55%|▌| 187/338 [00:00<00:00, 4924.50it/s, Materializing param=model.layers.15.self_attn.k_proj.bias]
Loading weights:  56%|▌| 188/338 [00:00<00:00, 4939.73it/s, Materializing param=model.layers.15.self_attn.k_proj.weigh
Loading weights:  56%|▌| 188/338 [00:00<00:00, 4933.02it/s, Materializing param=model.layers.15.self_attn.k_proj.weigh
Loading weights:  56%|▌| 189/338 [00:00<00:00, 4946.42it/s, Materializing param=model.layers.15.self_attn.o_proj.weigh
Loading weights:  56%|▌| 189/338 [00:00<00:00, 4939.89it/s, Materializing param=model.layers.15.self_attn.o_proj.weigh
Loading weights:  56%|▌| 190/338 [00:00<00:00, 4954.85it/s, Materializing param=model.layers.15.self_attn.q_proj.bias]
Loading weights:  56%|▌| 190/338 [00:00<00:00, 4948.14it/s, Materializing param=model.layers.15.self_attn.q_proj.bias]
Loading weights:  57%|▌| 191/338 [00:00<00:00, 4955.63it/s, Materializing param=model.layers.15.self_attn.q_proj.weigh
Loading weights:  57%|▌| 191/338 [00:00<00:00, 4948.77it/s, Materializing param=model.layers.15.self_attn.q_proj.weigh
Loading weights:  57%|▌| 192/338 [00:00<00:00, 4963.18it/s, Materializing param=model.layers.15.self_attn.v_proj.bias]
Loading weights:  57%|▌| 192/338 [00:00<00:00, 4956.65it/s, Materializing param=model.layers.15.self_attn.v_proj.bias]
Loading weights:  57%|▌| 193/338 [00:00<00:00, 4971.26it/s, Materializing param=model.layers.15.self_attn.v_proj.weigh
Loading weights:  57%|▌| 193/338 [00:00<00:00, 4964.65it/s, Materializing param=model.layers.15.self_attn.v_proj.weigh
Loading weights:  57%|▌| 194/338 [00:00<00:00, 4940.80it/s, Materializing param=model.layers.16.input_layernorm.weight
Loading weights:  57%|▌| 194/338 [00:00<00:00, 4934.00it/s, Materializing param=model.layers.16.input_layernorm.weight
Loading weights:  58%|█▏| 195/338 [00:00<00:00, 4946.44it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  58%|█▏| 195/338 [00:00<00:00, 4939.96it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  58%|█▏| 196/338 [00:00<00:00, 4954.31it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  58%|█▏| 196/338 [00:00<00:00, 4947.78it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  58%|██▎ | 197/338 [00:00<00:00, 4960.43it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  58%|██▎ | 197/338 [00:00<00:00, 4953.88it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  59%|▌| 198/338 [00:00<00:00, 4968.22it/s, Materializing param=model.layers.16.post_attention_layerno
Loading weights:  59%|▌| 198/338 [00:00<00:00, 4961.48it/s, Materializing param=model.layers.16.post_attention_layerno
Loading weights:  59%|▌| 199/338 [00:00<00:00, 4973.82it/s, Materializing param=model.layers.16.self_attn.k_proj.bias]
Loading weights:  59%|▌| 199/338 [00:00<00:00, 4967.40it/s, Materializing param=model.layers.16.self_attn.k_proj.bias]
Loading weights:  59%|▌| 200/338 [00:00<00:00, 4981.57it/s, Materializing param=model.layers.16.self_attn.k_proj.weigh
Loading weights:  59%|▌| 200/338 [00:00<00:00, 4974.98it/s, Materializing param=model.layers.16.self_attn.k_proj.weigh
Loading weights:  59%|▌| 201/338 [00:00<00:00, 4987.43it/s, Materializing param=model.layers.16.self_attn.o_proj.weigh
Loading weights:  59%|▌| 201/338 [00:00<00:00, 4980.74it/s, Materializing param=model.layers.16.self_attn.o_proj.weigh
Loading weights:  60%|▌| 202/338 [00:00<00:00, 4994.78it/s, Materializing param=model.layers.16.self_attn.q_proj.bias]
Loading weights:  60%|▌| 202/338 [00:00<00:00, 4988.13it/s, Materializing param=model.layers.16.self_attn.q_proj.bias]
Loading weights:  60%|▌| 203/338 [00:00<00:00, 5000.58it/s, Materializing param=model.layers.16.self_attn.q_proj.weigh
Loading weights:  60%|▌| 203/338 [00:00<00:00, 4994.01it/s, Materializing param=model.layers.16.self_attn.q_proj.weigh
Loading weights:  60%|▌| 204/338 [00:00<00:00, 5008.12it/s, Materializing param=model.layers.16.self_attn.v_proj.bias]
Loading weights:  60%|▌| 204/338 [00:00<00:00, 5001.51it/s, Materializing param=model.layers.16.self_attn.v_proj.bias]
Loading weights:  61%|▌| 205/338 [00:00<00:00, 5013.48it/s, Materializing param=model.layers.16.self_attn.v_proj.weigh
Loading weights:  61%|▌| 205/338 [00:00<00:00, 5007.23it/s, Materializing param=model.layers.16.self_attn.v_proj.weigh
Loading weights:  61%|▌| 206/338 [00:00<00:00, 5021.10it/s, Materializing param=model.layers.17.input_layernorm.weight
Loading weights:  61%|▌| 206/338 [00:00<00:00, 5014.87it/s, Materializing param=model.layers.17.input_layernorm.weight
Loading weights:  61%|█▏| 207/338 [00:00<00:00, 5027.86it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  61%|█▏| 207/338 [00:00<00:00, 5021.69it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  62%|█▏| 208/338 [00:00<00:00, 5035.30it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  62%|█▏| 208/338 [00:00<00:00, 5029.03it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  62%|██▍ | 209/338 [00:00<00:00, 5041.75it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  62%|██▍ | 209/338 [00:00<00:00, 5035.41it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  62%|▌| 210/338 [00:00<00:00, 5048.80it/s, Materializing param=model.layers.17.post_attention_layerno
Loading weights:  62%|▌| 210/338 [00:00<00:00, 5042.30it/s, Materializing param=model.layers.17.post_attention_layerno
Loading weights:  62%|▌| 211/338 [00:00<00:00, 5054.76it/s, Materializing param=model.layers.17.self_attn.k_proj.bias]
Loading weights:  62%|▌| 211/338 [00:00<00:00, 5048.36it/s, Materializing param=model.layers.17.self_attn.k_proj.bias]
Loading weights:  63%|▋| 212/338 [00:00<00:00, 5061.92it/s, Materializing param=model.layers.17.self_attn.k_proj.weigh
Loading weights:  63%|▋| 212/338 [00:00<00:00, 5055.36it/s, Materializing param=model.layers.17.self_attn.k_proj.weigh
Loading weights:  63%|▋| 213/338 [00:00<00:00, 5067.97it/s, Materializing param=model.layers.17.self_attn.o_proj.weigh
Loading weights:  63%|▋| 213/338 [00:00<00:00, 5061.71it/s, Materializing param=model.layers.17.self_attn.o_proj.weigh
Loading weights:  63%|▋| 214/338 [00:00<00:00, 5075.09it/s, Materializing param=model.layers.17.self_attn.q_proj.bias]
Loading weights:  63%|▋| 214/338 [00:00<00:00, 5068.99it/s, Materializing param=model.layers.17.self_attn.q_proj.bias]
Loading weights:  64%|▋| 215/338 [00:00<00:00, 5081.51it/s, Materializing param=model.layers.17.self_attn.q_proj.weigh
Loading weights:  64%|▋| 215/338 [00:00<00:00, 5075.25it/s, Materializing param=model.layers.17.self_attn.q_proj.weigh
Loading weights:  64%|▋| 216/338 [00:00<00:00, 5088.29it/s, Materializing param=model.layers.17.self_attn.v_proj.bias]
Loading weights:  64%|▋| 216/338 [00:00<00:00, 5082.01it/s, Materializing param=model.layers.17.self_attn.v_proj.bias]
Loading weights:  64%|▋| 217/338 [00:00<00:00, 5094.28it/s, Materializing param=model.layers.17.self_attn.v_proj.weigh
Loading weights:  64%|▋| 217/338 [00:00<00:00, 5087.84it/s, Materializing param=model.layers.17.self_attn.v_proj.weigh
Loading weights:  64%|▋| 218/338 [00:00<00:00, 5065.00it/s, Materializing param=model.layers.18.input_layernorm.weight
Loading weights:  64%|▋| 218/338 [00:00<00:00, 5058.69it/s, Materializing param=model.layers.18.input_layernorm.weight
Loading weights:  65%|█▎| 219/338 [00:00<00:00, 5071.37it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  65%|█▎| 219/338 [00:00<00:00, 5064.86it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  65%|█▎| 220/338 [00:00<00:00, 5053.02it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  65%|█▎| 220/338 [00:00<00:00, 5046.58it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  65%|██▌ | 221/338 [00:00<00:00, 5057.27it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  65%|██▌ | 221/338 [00:00<00:00, 5051.15it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  66%|▋| 222/338 [00:00<00:00, 5063.55it/s, Materializing param=model.layers.18.post_attention_layerno
Loading weights:  66%|▋| 222/338 [00:00<00:00, 5057.36it/s, Materializing param=model.layers.18.post_attention_layerno
Loading weights:  66%|▋| 223/338 [00:00<00:00, 5017.81it/s, Materializing param=model.layers.18.self_attn.k_proj.bias]
Loading weights:  66%|▋| 223/338 [00:00<00:00, 5011.33it/s, Materializing param=model.layers.18.self_attn.k_proj.bias]
Loading weights:  66%|▋| 224/338 [00:00<00:00, 5023.76it/s, Materializing param=model.layers.18.self_attn.k_proj.weigh
Loading weights:  66%|▋| 224/338 [00:00<00:00, 5017.94it/s, Materializing param=model.layers.18.self_attn.k_proj.weigh
Loading weights:  67%|▋| 225/338 [00:00<00:00, 5027.05it/s, Materializing param=model.layers.18.self_attn.o_proj.weigh
Loading weights:  67%|▋| 225/338 [00:00<00:00, 5021.03it/s, Materializing param=model.layers.18.self_attn.o_proj.weigh
Loading weights:  67%|▋| 226/338 [00:00<00:00, 5033.63it/s, Materializing param=model.layers.18.self_attn.q_proj.bias]
Loading weights:  67%|▋| 226/338 [00:00<00:00, 5027.97it/s, Materializing param=model.layers.18.self_attn.q_proj.bias]
Loading weights:  67%|▋| 227/338 [00:00<00:00, 5039.28it/s, Materializing param=model.layers.18.self_attn.q_proj.weigh
Loading weights:  67%|▋| 227/338 [00:00<00:00, 5033.53it/s, Materializing param=model.layers.18.self_attn.q_proj.weigh
Loading weights:  67%|▋| 228/338 [00:00<00:00, 5046.31it/s, Materializing param=model.layers.18.self_attn.v_proj.bias]
Loading weights:  67%|▋| 228/338 [00:00<00:00, 5040.65it/s, Materializing param=model.layers.18.self_attn.v_proj.bias]
Loading weights:  68%|▋| 229/338 [00:00<00:00, 5051.76it/s, Materializing param=model.layers.18.self_attn.v_proj.weigh
Loading weights:  68%|▋| 229/338 [00:00<00:00, 5046.10it/s, Materializing param=model.layers.18.self_attn.v_proj.weigh
Loading weights:  68%|▋| 230/338 [00:00<00:00, 5058.31it/s, Materializing param=model.layers.19.input_layernorm.weight
Loading weights:  68%|▋| 230/338 [00:00<00:00, 5052.48it/s, Materializing param=model.layers.19.input_layernorm.weight
Loading weights:  68%|█▎| 231/338 [00:00<00:00, 5043.83it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  68%|█▎| 231/338 [00:00<00:00, 5037.82it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  69%|█▎| 232/338 [00:00<00:00, 5048.42it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  69%|█▎| 232/338 [00:00<00:00, 5042.67it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  69%|██▊ | 233/338 [00:00<00:00, 5054.76it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  69%|██▊ | 233/338 [00:00<00:00, 5048.97it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  69%|▋| 234/338 [00:00<00:00, 5059.42it/s, Materializing param=model.layers.19.post_attention_layerno
Loading weights:  69%|▋| 234/338 [00:00<00:00, 5053.46it/s, Materializing param=model.layers.19.post_attention_layerno
Loading weights:  70%|▋| 235/338 [00:00<00:00, 5025.12it/s, Materializing param=model.layers.19.self_attn.k_proj.bias]
Loading weights:  70%|▋| 235/338 [00:00<00:00, 5019.05it/s, Materializing param=model.layers.19.self_attn.k_proj.bias]
Loading weights:  70%|▋| 236/338 [00:00<00:00, 5030.24it/s, Materializing param=model.layers.19.self_attn.k_proj.weigh
Loading weights:  70%|▋| 236/338 [00:00<00:00, 5024.19it/s, Materializing param=model.layers.19.self_attn.k_proj.weigh
Loading weights:  70%|▋| 237/338 [00:00<00:00, 5003.83it/s, Materializing param=model.layers.19.self_attn.o_proj.weigh
Loading weights:  70%|▋| 237/338 [00:00<00:00, 4997.94it/s, Materializing param=model.layers.19.self_attn.o_proj.weigh
Loading weights:  70%|▋| 238/338 [00:00<00:00, 5009.51it/s, Materializing param=model.layers.19.self_attn.q_proj.bias]
Loading weights:  70%|▋| 238/338 [00:00<00:00, 5004.11it/s, Materializing param=model.layers.19.self_attn.q_proj.bias]
Loading weights:  71%|▋| 239/338 [00:00<00:00, 5014.55it/s, Materializing param=model.layers.19.self_attn.q_proj.weigh
Loading weights:  71%|▋| 239/338 [00:00<00:00, 5008.86it/s, Materializing param=model.layers.19.self_attn.q_proj.weigh
Loading weights:  71%|▋| 240/338 [00:00<00:00, 5020.39it/s, Materializing param=model.layers.19.self_attn.v_proj.bias]
Loading weights:  71%|▋| 240/338 [00:00<00:00, 5014.91it/s, Materializing param=model.layers.19.self_attn.v_proj.bias]
Loading weights:  71%|▋| 241/338 [00:00<00:00, 5009.58it/s, Materializing param=model.layers.19.self_attn.v_proj.weigh
Loading weights:  71%|▋| 241/338 [00:00<00:00, 5003.80it/s, Materializing param=model.layers.19.self_attn.v_proj.weigh
Loading weights:  72%|▋| 242/338 [00:00<00:00, 5013.67it/s, Materializing param=model.layers.20.input_layernorm.weight
Loading weights:  72%|▋| 242/338 [00:00<00:00, 5008.07it/s, Materializing param=model.layers.20.input_layernorm.weight
Loading weights:  72%|█▍| 243/338 [00:00<00:00, 5019.58it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  72%|█▍| 243/338 [00:00<00:00, 5013.98it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  72%|█▍| 244/338 [00:00<00:00, 5023.83it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  72%|█▍| 244/338 [00:00<00:00, 5018.41it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  72%|██▉ | 245/338 [00:00<00:00, 5030.20it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  72%|██▉ | 245/338 [00:00<00:00, 5024.89it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  73%|▋| 246/338 [00:00<00:00, 5035.13it/s, Materializing param=model.layers.20.post_attention_layerno
Loading weights:  73%|▋| 246/338 [00:00<00:00, 5029.63it/s, Materializing param=model.layers.20.post_attention_layerno
Loading weights:  73%|▋| 247/338 [00:00<00:00, 5040.81it/s, Materializing param=model.layers.20.self_attn.k_proj.bias]
Loading weights:  73%|▋| 247/338 [00:00<00:00, 5035.23it/s, Materializing param=model.layers.20.self_attn.k_proj.bias]
Loading weights:  73%|▋| 248/338 [00:00<00:00, 5045.09it/s, Materializing param=model.layers.20.self_attn.k_proj.weigh
Loading weights:  73%|▋| 248/338 [00:00<00:00, 5039.67it/s, Materializing param=model.layers.20.self_attn.k_proj.weigh
Loading weights:  74%|▋| 249/338 [00:00<00:00, 5028.49it/s, Materializing param=model.layers.20.self_attn.o_proj.weigh
Loading weights:  74%|▋| 249/338 [00:00<00:00, 5022.93it/s, Materializing param=model.layers.20.self_attn.o_proj.weigh
Loading weights:  74%|▋| 250/338 [00:00<00:00, 5033.92it/s, Materializing param=model.layers.20.self_attn.q_proj.bias]
Loading weights:  74%|▋| 250/338 [00:00<00:00, 5028.88it/s, Materializing param=model.layers.20.self_attn.q_proj.bias]
Loading weights:  74%|▋| 251/338 [00:00<00:00, 5040.27it/s, Materializing param=model.layers.20.self_attn.q_proj.weigh
Loading weights:  74%|▋| 251/338 [00:00<00:00, 5035.11it/s, Materializing param=model.layers.20.self_attn.q_proj.weigh
Loading weights:  75%|▋| 252/338 [00:00<00:00, 5036.07it/s, Materializing param=model.layers.20.self_attn.v_proj.bias]
Loading weights:  75%|▋| 252/338 [00:00<00:00, 5030.55it/s, Materializing param=model.layers.20.self_attn.v_proj.bias]
Loading weights:  75%|▋| 253/338 [00:00<00:00, 5041.30it/s, Materializing param=model.layers.20.self_attn.v_proj.weigh
Loading weights:  75%|▋| 253/338 [00:00<00:00, 5035.97it/s, Materializing param=model.layers.20.self_attn.v_proj.weigh
Loading weights:  75%|▊| 254/338 [00:00<00:00, 5046.15it/s, Materializing param=model.layers.21.input_layernorm.weight
Loading weights:  75%|▊| 254/338 [00:00<00:00, 5040.54it/s, Materializing param=model.layers.21.input_layernorm.weight
Loading weights:  75%|█▌| 255/338 [00:00<00:00, 5027.96it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  75%|█▌| 255/338 [00:00<00:00, 5022.55it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  76%|█▌| 256/338 [00:00<00:00, 5033.10it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  76%|█▌| 256/338 [00:00<00:00, 5027.96it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  76%|███ | 257/338 [00:00<00:00, 5037.46it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  76%|███ | 257/338 [00:00<00:00, 5032.31it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  76%|▊| 258/338 [00:00<00:00, 5014.65it/s, Materializing param=model.layers.21.post_attention_layerno
Loading weights:  76%|▊| 258/338 [00:00<00:00, 5009.03it/s, Materializing param=model.layers.21.post_attention_layerno
Loading weights:  77%|▊| 259/338 [00:00<00:00, 5019.87it/s, Materializing param=model.layers.21.self_attn.k_proj.bias]
Loading weights:  77%|▊| 259/338 [00:00<00:00, 5014.91it/s, Materializing param=model.layers.21.self_attn.k_proj.bias]
Loading weights:  77%|▊| 260/338 [00:00<00:00, 5024.30it/s, Materializing param=model.layers.21.self_attn.k_proj.weigh
Loading weights:  77%|▊| 260/338 [00:00<00:00, 5019.37it/s, Materializing param=model.layers.21.self_attn.k_proj.weigh
Loading weights:  77%|▊| 261/338 [00:00<00:00, 5030.32it/s, Materializing param=model.layers.21.self_attn.o_proj.weigh
Loading weights:  77%|▊| 261/338 [00:00<00:00, 5025.24it/s, Materializing param=model.layers.21.self_attn.o_proj.weigh
Loading weights:  78%|▊| 262/338 [00:00<00:00, 5018.65it/s, Materializing param=model.layers.21.self_attn.q_proj.bias]
Loading weights:  78%|▊| 262/338 [00:00<00:00, 5013.42it/s, Materializing param=model.layers.21.self_attn.q_proj.bias]
Loading weights:  78%|▊| 263/338 [00:00<00:00, 5002.59it/s, Materializing param=model.layers.21.self_attn.q_proj.weigh
Loading weights:  78%|▊| 263/338 [00:00<00:00, 4997.36it/s, Materializing param=model.layers.21.self_attn.q_proj.weigh
Loading weights:  78%|▊| 264/338 [00:00<00:00, 5008.12it/s, Materializing param=model.layers.21.self_attn.v_proj.bias]
Loading weights:  78%|▊| 264/338 [00:00<00:00, 5003.42it/s, Materializing param=model.layers.21.self_attn.v_proj.bias]
Loading weights:  78%|▊| 265/338 [00:00<00:00, 5012.86it/s, Materializing param=model.layers.21.self_attn.v_proj.weigh
Loading weights:  78%|▊| 265/338 [00:00<00:00, 5007.91it/s, Materializing param=model.layers.21.self_attn.v_proj.weigh
Loading weights:  79%|▊| 266/338 [00:00<00:00, 5018.46it/s, Materializing param=model.layers.22.input_layernorm.weight
Loading weights:  79%|▊| 266/338 [00:00<00:00, 5013.62it/s, Materializing param=model.layers.22.input_layernorm.weight
Loading weights:  79%|█▌| 267/338 [00:00<00:00, 5023.10it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  79%|█▌| 267/338 [00:00<00:00, 5018.12it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  79%|█▌| 268/338 [00:00<00:00, 5028.89it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  79%|█▌| 268/338 [00:00<00:00, 5024.15it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  80%|███▏| 269/338 [00:00<00:00, 5026.18it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  80%|███▏| 269/338 [00:00<00:00, 5021.11it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  80%|▊| 270/338 [00:00<00:00, 5029.50it/s, Materializing param=model.layers.22.post_attention_layerno
Loading weights:  80%|▊| 270/338 [00:00<00:00, 5024.05it/s, Materializing param=model.layers.22.post_attention_layerno
Loading weights:  80%|▊| 271/338 [00:00<00:00, 4993.99it/s, Materializing param=model.layers.22.self_attn.k_proj.bias]
Loading weights:  80%|▊| 271/338 [00:00<00:00, 4988.70it/s, Materializing param=model.layers.22.self_attn.k_proj.bias]
Loading weights:  80%|▊| 272/338 [00:00<00:00, 4998.62it/s, Materializing param=model.layers.22.self_attn.k_proj.weigh
Loading weights:  80%|▊| 272/338 [00:00<00:00, 4993.13it/s, Materializing param=model.layers.22.self_attn.k_proj.weigh
Loading weights:  81%|▊| 273/338 [00:00<00:00, 5000.28it/s, Materializing param=model.layers.22.self_attn.o_proj.weigh
Loading weights:  81%|▊| 273/338 [00:00<00:00, 4995.64it/s, Materializing param=model.layers.22.self_attn.o_proj.weigh
Loading weights:  81%|▊| 274/338 [00:00<00:00, 5006.01it/s, Materializing param=model.layers.22.self_attn.q_proj.bias]
Loading weights:  81%|▊| 274/338 [00:00<00:00, 5001.56it/s, Materializing param=model.layers.22.self_attn.q_proj.bias]
Loading weights:  81%|▊| 275/338 [00:00<00:00, 5001.40it/s, Materializing param=model.layers.22.self_attn.q_proj.weigh
Loading weights:  81%|▊| 275/338 [00:00<00:00, 4996.36it/s, Materializing param=model.layers.22.self_attn.q_proj.weigh
Loading weights:  82%|▊| 276/338 [00:00<00:00, 5005.29it/s, Materializing param=model.layers.22.self_attn.v_proj.bias]
Loading weights:  82%|▊| 276/338 [00:00<00:00, 5000.64it/s, Materializing param=model.layers.22.self_attn.v_proj.bias]
Loading weights:  82%|▊| 277/338 [00:00<00:00, 5010.99it/s, Materializing param=model.layers.22.self_attn.v_proj.weigh
Loading weights:  82%|▊| 277/338 [00:00<00:00, 5006.34it/s, Materializing param=model.layers.22.self_attn.v_proj.weigh
Loading weights:  82%|▊| 278/338 [00:00<00:00, 5007.26it/s, Materializing param=model.layers.23.input_layernorm.weight
Loading weights:  82%|▊| 278/338 [00:00<00:00, 5002.41it/s, Materializing param=model.layers.23.input_layernorm.weight
Loading weights:  83%|█▋| 279/338 [00:00<00:00, 5011.39it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  83%|█▋| 279/338 [00:00<00:00, 5006.83it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  83%|█▋| 280/338 [00:00<00:00, 5016.98it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  83%|█▋| 280/338 [00:00<00:00, 5012.42it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  83%|███▎| 281/338 [00:00<00:00, 5021.32it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  83%|███▎| 281/338 [00:00<00:00, 5016.58it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  83%|▊| 282/338 [00:00<00:00, 5026.85it/s, Materializing param=model.layers.23.post_attention_layerno
Loading weights:  83%|▊| 282/338 [00:00<00:00, 5022.16it/s, Materializing param=model.layers.23.post_attention_layerno
Loading weights:  84%|▊| 283/338 [00:00<00:00, 5030.91it/s, Materializing param=model.layers.23.self_attn.k_proj.bias]
Loading weights:  84%|▊| 283/338 [00:00<00:00, 5026.18it/s, Materializing param=model.layers.23.self_attn.k_proj.bias]
Loading weights:  84%|▊| 284/338 [00:00<00:00, 5036.09it/s, Materializing param=model.layers.23.self_attn.k_proj.weigh
Loading weights:  84%|▊| 284/338 [00:00<00:00, 5031.54it/s, Materializing param=model.layers.23.self_attn.k_proj.weigh
Loading weights:  84%|▊| 285/338 [00:00<00:00, 5040.53it/s, Materializing param=model.layers.23.self_attn.o_proj.weigh
Loading weights:  84%|▊| 285/338 [00:00<00:00, 5036.03it/s, Materializing param=model.layers.23.self_attn.o_proj.weigh
Loading weights:  85%|▊| 286/338 [00:00<00:00, 5005.55it/s, Materializing param=model.layers.23.self_attn.q_proj.bias]
Loading weights:  85%|▊| 286/338 [00:00<00:00, 5000.55it/s, Materializing param=model.layers.23.self_attn.q_proj.bias]
Loading weights:  85%|▊| 287/338 [00:00<00:00, 5010.11it/s, Materializing param=model.layers.23.self_attn.q_proj.weigh
Loading weights:  85%|▊| 287/338 [00:00<00:00, 5005.68it/s, Materializing param=model.layers.23.self_attn.q_proj.weigh
Loading weights:  85%|▊| 288/338 [00:00<00:00, 5014.38it/s, Materializing param=model.layers.23.self_attn.v_proj.bias]
Loading weights:  85%|▊| 288/338 [00:00<00:00, 5009.74it/s, Materializing param=model.layers.23.self_attn.v_proj.bias]
Loading weights:  86%|▊| 289/338 [00:00<00:00, 4996.43it/s, Materializing param=model.layers.23.self_attn.v_proj.weigh
Loading weights:  86%|▊| 289/338 [00:00<00:00, 4991.72it/s, Materializing param=model.layers.23.self_attn.v_proj.weigh
Loading weights:  86%|▊| 290/338 [00:00<00:00, 5001.35it/s, Materializing param=model.layers.24.input_layernorm.weight
Loading weights:  86%|▊| 290/338 [00:00<00:00, 4996.91it/s, Materializing param=model.layers.24.input_layernorm.weight
Loading weights:  86%|█▋| 291/338 [00:00<00:00, 5005.57it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  86%|█▋| 291/338 [00:00<00:00, 5001.16it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  86%|█▋| 292/338 [00:00<00:00, 5010.99it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  86%|█▋| 292/338 [00:00<00:00, 5006.51it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  87%|███▍| 293/338 [00:00<00:00, 5014.90it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  87%|███▍| 293/338 [00:00<00:00, 5010.24it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  87%|▊| 294/338 [00:00<00:00, 5019.87it/s, Materializing param=model.layers.24.post_attention_layerno
Loading weights:  87%|▊| 294/338 [00:00<00:00, 5015.23it/s, Materializing param=model.layers.24.post_attention_layerno
Loading weights:  87%|▊| 295/338 [00:00<00:00, 5006.70it/s, Materializing param=model.layers.24.self_attn.k_proj.bias]
Loading weights:  87%|▊| 295/338 [00:00<00:00, 5002.14it/s, Materializing param=model.layers.24.self_attn.k_proj.bias]
Loading weights:  88%|▉| 296/338 [00:00<00:00, 5010.51it/s, Materializing param=model.layers.24.self_attn.k_proj.weigh
Loading weights:  88%|▉| 296/338 [00:00<00:00, 5006.00it/s, Materializing param=model.layers.24.self_attn.k_proj.weigh
Loading weights:  88%|▉| 297/338 [00:00<00:00, 5015.43it/s, Materializing param=model.layers.24.self_attn.o_proj.weigh
Loading weights:  88%|▉| 297/338 [00:00<00:00, 5010.77it/s, Materializing param=model.layers.24.self_attn.o_proj.weigh
Loading weights:  88%|▉| 298/338 [00:00<00:00, 4984.68it/s, Materializing param=model.layers.24.self_attn.q_proj.bias]
Loading weights:  88%|▉| 298/338 [00:00<00:00, 4980.05it/s, Materializing param=model.layers.24.self_attn.q_proj.bias]
Loading weights:  88%|▉| 299/338 [00:00<00:00, 4989.21it/s, Materializing param=model.layers.24.self_attn.q_proj.weigh
Loading weights:  88%|▉| 299/338 [00:00<00:00, 4984.72it/s, Materializing param=model.layers.24.self_attn.q_proj.weigh
Loading weights:  89%|▉| 300/338 [00:00<00:00, 4993.10it/s, Materializing param=model.layers.24.self_attn.v_proj.bias]
Loading weights:  89%|▉| 300/338 [00:00<00:00, 4988.80it/s, Materializing param=model.layers.24.self_attn.v_proj.bias]
Loading weights:  89%|▉| 301/338 [00:00<00:00, 4998.14it/s, Materializing param=model.layers.24.self_attn.v_proj.weigh
Loading weights:  89%|▉| 301/338 [00:00<00:00, 4993.77it/s, Materializing param=model.layers.24.self_attn.v_proj.weigh
Loading weights:  89%|▉| 302/338 [00:00<00:00, 5002.03it/s, Materializing param=model.layers.25.input_layernorm.weight
Loading weights:  89%|▉| 302/338 [00:00<00:00, 4997.79it/s, Materializing param=model.layers.25.input_layernorm.weight
Loading weights:  90%|█▊| 303/338 [00:00<00:00, 5007.01it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  90%|█▊| 303/338 [00:00<00:00, 5002.44it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  90%|█▊| 304/338 [00:00<00:00, 4989.82it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  90%|█▊| 304/338 [00:00<00:00, 4985.23it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  90%|███▌| 305/338 [00:00<00:00, 4993.02it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  90%|███▌| 305/338 [00:00<00:00, 4988.55it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  91%|▉| 306/338 [00:00<00:00, 4998.00it/s, Materializing param=model.layers.25.post_attention_layerno
Loading weights:  91%|▉| 306/338 [00:00<00:00, 4993.61it/s, Materializing param=model.layers.25.post_attention_layerno
Loading weights:  91%|▉| 307/338 [00:00<00:00, 5001.91it/s, Materializing param=model.layers.25.self_attn.k_proj.bias]
Loading weights:  91%|▉| 307/338 [00:00<00:00, 4997.64it/s, Materializing param=model.layers.25.self_attn.k_proj.bias]
Loading weights:  91%|▉| 308/338 [00:00<00:00, 5006.84it/s, Materializing param=model.layers.25.self_attn.k_proj.weigh
Loading weights:  91%|▉| 308/338 [00:00<00:00, 5002.58it/s, Materializing param=model.layers.25.self_attn.k_proj.weigh
Loading weights:  91%|▉| 309/338 [00:00<00:00, 4991.37it/s, Materializing param=model.layers.25.self_attn.o_proj.weigh
Loading weights:  91%|▉| 309/338 [00:00<00:00, 4986.71it/s, Materializing param=model.layers.25.self_attn.o_proj.weigh
Loading weights:  92%|▉| 310/338 [00:00<00:00, 4995.77it/s, Materializing param=model.layers.25.self_attn.q_proj.bias]
Loading weights:  92%|▉| 310/338 [00:00<00:00, 4991.76it/s, Materializing param=model.layers.25.self_attn.q_proj.bias]
Loading weights:  92%|▉| 311/338 [00:00<00:00, 4966.26it/s, Materializing param=model.layers.25.self_attn.q_proj.weigh
Loading weights:  92%|▉| 311/338 [00:00<00:00, 4961.65it/s, Materializing param=model.layers.25.self_attn.q_proj.weigh
Loading weights:  92%|▉| 312/338 [00:00<00:00, 4970.44it/s, Materializing param=model.layers.25.self_attn.v_proj.bias]
Loading weights:  92%|▉| 312/338 [00:00<00:00, 4966.29it/s, Materializing param=model.layers.25.self_attn.v_proj.bias]
Loading weights:  93%|▉| 313/338 [00:00<00:00, 4975.34it/s, Materializing param=model.layers.25.self_attn.v_proj.weigh
Loading weights:  93%|▉| 313/338 [00:00<00:00, 4971.36it/s, Materializing param=model.layers.25.self_attn.v_proj.weigh
Loading weights:  93%|▉| 314/338 [00:00<00:00, 4977.91it/s, Materializing param=model.layers.26.input_layernorm.weight
Loading weights:  93%|▉| 314/338 [00:00<00:00, 4973.85it/s, Materializing param=model.layers.26.input_layernorm.weight
Loading weights:  93%|█▊| 315/338 [00:00<00:00, 4982.73it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  93%|█▊| 315/338 [00:00<00:00, 4978.77it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  93%|█▊| 316/338 [00:00<00:00, 4986.66it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  93%|█▊| 316/338 [00:00<00:00, 4982.54it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  94%|███▊| 317/338 [00:00<00:00, 4991.68it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  94%|███▊| 317/338 [00:00<00:00, 4987.64it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  94%|▉| 318/338 [00:00<00:00, 4995.48it/s, Materializing param=model.layers.26.post_attention_layerno
Loading weights:  94%|▉| 318/338 [00:00<00:00, 4991.31it/s, Materializing param=model.layers.26.post_attention_layerno
Loading weights:  94%|▉| 319/338 [00:00<00:00, 5000.24it/s, Materializing param=model.layers.26.self_attn.k_proj.bias]
Loading weights:  94%|▉| 319/338 [00:00<00:00, 4996.22it/s, Materializing param=model.layers.26.self_attn.k_proj.bias]
Loading weights:  95%|▉| 320/338 [00:00<00:00, 4977.61it/s, Materializing param=model.layers.26.self_attn.k_proj.weigh
Loading weights:  95%|▉| 320/338 [00:00<00:00, 4972.98it/s, Materializing param=model.layers.26.self_attn.k_proj.weigh
Loading weights:  95%|▉| 321/338 [00:00<00:00, 4981.54it/s, Materializing param=model.layers.26.self_attn.o_proj.weigh
Loading weights:  95%|▉| 321/338 [00:00<00:00, 4977.49it/s, Materializing param=model.layers.26.self_attn.o_proj.weigh
Loading weights:  95%|▉| 322/338 [00:00<00:00, 4986.10it/s, Materializing param=model.layers.26.self_attn.q_proj.bias]
Loading weights:  95%|▉| 322/338 [00:00<00:00, 4982.15it/s, Materializing param=model.layers.26.self_attn.q_proj.bias]
Loading weights:  96%|▉| 323/338 [00:00<00:00, 4983.80it/s, Materializing param=model.layers.26.self_attn.q_proj.weigh
Loading weights:  96%|▉| 323/338 [00:00<00:00, 4979.75it/s, Materializing param=model.layers.26.self_attn.q_proj.weigh
Loading weights:  96%|▉| 324/338 [00:00<00:00, 4988.71it/s, Materializing param=model.layers.26.self_attn.v_proj.bias]
Loading weights:  96%|▉| 324/338 [00:00<00:00, 4984.79it/s, Materializing param=model.layers.26.self_attn.v_proj.bias]
Loading weights:  96%|▉| 325/338 [00:00<00:00, 4993.58it/s, Materializing param=model.layers.26.self_attn.v_proj.weigh
Loading weights:  96%|▉| 325/338 [00:00<00:00, 4989.62it/s, Materializing param=model.layers.26.self_attn.v_proj.weigh
Loading weights:  96%|▉| 326/338 [00:00<00:00, 4998.46it/s, Materializing param=model.layers.27.input_layernorm.weight
Loading weights:  96%|▉| 326/338 [00:00<00:00, 4994.44it/s, Materializing param=model.layers.27.input_layernorm.weight
Loading weights:  97%|█▉| 327/338 [00:00<00:00, 5000.06it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  97%|█▉| 327/338 [00:00<00:00, 4996.02it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  97%|█▉| 328/338 [00:00<00:00, 5004.84it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  97%|█▉| 328/338 [00:00<00:00, 5000.88it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  97%|███▉| 329/338 [00:00<00:00, 5007.44it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  97%|███▉| 329/338 [00:00<00:00, 5003.41it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  98%|▉| 330/338 [00:00<00:00, 5012.66it/s, Materializing param=model.layers.27.post_attention_layerno
Loading weights:  98%|▉| 330/338 [00:00<00:00, 5008.83it/s, Materializing param=model.layers.27.post_attention_layerno
Loading weights:  98%|▉| 331/338 [00:00<00:00, 5018.11it/s, Materializing param=model.layers.27.self_attn.k_proj.bias]
Loading weights:  98%|▉| 331/338 [00:00<00:00, 5014.34it/s, Materializing param=model.layers.27.self_attn.k_proj.bias]
Loading weights:  98%|▉| 332/338 [00:00<00:00, 5023.63it/s, Materializing param=model.layers.27.self_attn.k_proj.weigh
Loading weights:  98%|▉| 332/338 [00:00<00:00, 5019.77it/s, Materializing param=model.layers.27.self_attn.k_proj.weigh
Loading weights:  99%|▉| 333/338 [00:00<00:00, 5029.07it/s, Materializing param=model.layers.27.self_attn.o_proj.weigh
Loading weights:  99%|▉| 333/338 [00:00<00:00, 5025.31it/s, Materializing param=model.layers.27.self_attn.o_proj.weigh
Loading weights:  99%|▉| 334/338 [00:00<00:00, 5034.58it/s, Materializing param=model.layers.27.self_attn.q_proj.bias]
Loading weights:  99%|▉| 334/338 [00:00<00:00, 5030.91it/s, Materializing param=model.layers.27.self_attn.q_proj.bias]
Loading weights:  99%|▉| 335/338 [00:00<00:00, 5039.66it/s, Materializing param=model.layers.27.self_attn.q_proj.weigh
Loading weights:  99%|▉| 335/338 [00:00<00:00, 5035.92it/s, Materializing param=model.layers.27.self_attn.q_proj.weigh
Loading weights:  99%|▉| 336/338 [00:00<00:00, 5045.38it/s, Materializing param=model.layers.27.self_attn.v_proj.bias]
Loading weights:  99%|▉| 336/338 [00:00<00:00, 5041.79it/s, Materializing param=model.layers.27.self_attn.v_proj.bias]
Loading weights: 100%|▉| 337/338 [00:00<00:00, 5050.71it/s, Materializing param=model.layers.27.self_attn.v_proj.weigh
Loading weights: 100%|▉| 337/338 [00:00<00:00, 5046.99it/s, Materializing param=model.layers.27.self_attn.v_proj.weigh
Loading weights: 100%|█████████████████████| 338/338 [00:00<00:00, 5056.21it/s, Materializing param=model.norm.weight]
Loading weights: 100%|█████████████████████| 338/338 [00:00<00:00, 5052.44it/s, Materializing param=model.norm.weight]
Loading weights: 100%|█████████████████████| 338/338 [00:00<00:00, 5043.89it/s, Materializing param=model.norm.weight]
-[2026-02-10 14:54:56,103] [WARNING] [torchao.<module>:39] [PID:12849] Skipping import of cpp extensions due to incompatible torch version 2.9.1+cu128 for torchao version 0.13.0
-[2026-02-10 14:55:09,162] [WARNING] [accelerate.utils.dataclasses.__post_init__:1962] [PID:12849] sharding_strategy is deprecated in favor of reshard_after_forward. This will be removed in a future version of Accelerate.
-[2026-02-10 15:32:29,946] [WARNING] [py.warnings._showwarnmsg:110] [PID:12849] /root/miniconda3/envs/py3.11/lib/python3.11/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html .
+
Loading dataset from disk:   0%|                                                              | 0/208 [00:00<?, ?it/s]
Loading dataset from disk: 100%|████████████████████████████████████████████████| 208/208 [00:00<00:00, 237133.80it/s]
+
Loading weights:   0%|                                                                        | 0/146 [00:00<?, ?it/s]
Loading weights:   1%|               | 1/146 [00:00<00:00, 7423.55it/s, Materializing param=model.embed_tokens.weight]
Loading weights:   1%|               | 1/146 [00:00<00:00, 3728.27it/s, Materializing param=model.embed_tokens.weight]
Loading weights:   1%|   | 2/146 [00:00<00:00, 3985.09it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|   | 2/146 [00:00<00:00, 1457.11it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   2%|     | 3/146 [00:00<00:00, 1893.59it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   2%|     | 3/146 [00:00<00:00, 1740.13it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   3%|▏    | 4/146 [00:00<00:00, 2094.01it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   3%|▏    | 4/146 [00:00<00:00, 1979.85it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   3%|▏      | 5/146 [00:00<00:00, 2270.87it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   3%|▏      | 5/146 [00:00<00:00, 2033.11it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   4%| | 6/146 [00:00<00:00, 2282.41it/s, Materializing param=model.layers.0.post_attention_layernorm.
Loading weights:   4%| | 6/146 [00:00<00:00, 2201.54it/s, Materializing param=model.layers.0.post_attention_layernorm.
Loading weights:   5%|  | 7/146 [00:00<00:00, 2415.68it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   5%|  | 7/146 [00:00<00:00, 2336.66it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   5%|  | 8/146 [00:00<00:00, 2534.90it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   5%|  | 8/146 [00:00<00:00, 2457.48it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   6%|  | 9/146 [00:00<00:00, 2362.99it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   6%|  | 9/146 [00:00<00:00, 2304.98it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   7%| | 10/146 [00:00<00:00, 2443.09it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   7%| | 10/146 [00:00<00:00, 2386.79it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   8%|▏ | 11/146 [00:00<00:00, 2534.46it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   8%|▏ | 11/146 [00:00<00:00, 2477.17it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   8%|▎   | 12/146 [00:00<00:00, 2605.16it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   8%|▎   | 12/146 [00:00<00:00, 2554.13it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   9%|▎   | 13/146 [00:00<00:00, 2584.17it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   9%|▎   | 13/146 [00:00<00:00, 2533.38it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:  10%|▌     | 14/146 [00:00<00:00, 2450.35it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:  10%|▌     | 14/146 [00:00<00:00, 2408.44it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:  10%| | 15/146 [00:00<00:00, 2509.16it/s, Materializing param=model.layers.1.post_attention_layernorm
Loading weights:  10%| | 15/146 [00:00<00:00, 2468.59it/s, Materializing param=model.layers.1.post_attention_layernorm
Loading weights:  11%| | 16/146 [00:00<00:00, 2538.44it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:  11%| | 16/146 [00:00<00:00, 2320.90it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:  12%| | 17/146 [00:00<00:00, 2356.12it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:  12%| | 17/146 [00:00<00:00, 2325.61it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:  12%| | 18/146 [00:00<00:00, 2360.11it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:  12%| | 18/146 [00:00<00:00, 2295.24it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:  13%|▏| 19/146 [00:00<00:00, 2315.41it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:  13%|▏| 19/146 [00:00<00:00, 2248.07it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:  14%|▎ | 20/146 [00:00<00:00, 2114.44it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:  14%|▎ | 20/146 [00:00<00:00, 2091.61it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:  14%|▌   | 21/146 [00:00<00:00, 2156.14it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:  14%|▌   | 21/146 [00:00<00:00, 2133.63it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:  15%|▌   | 22/146 [00:00<00:00, 2176.34it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:  15%|▌   | 22/146 [00:00<00:00, 2155.95it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:  16%|▉     | 23/146 [00:00<00:00, 2219.87it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:  16%|▉     | 23/146 [00:00<00:00, 2199.93it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:  16%|▏| 24/146 [00:00<00:00, 2259.10it/s, Materializing param=model.layers.2.post_attention_layernorm
Loading weights:  16%|▏| 24/146 [00:00<00:00, 2236.47it/s, Materializing param=model.layers.2.post_attention_layernorm
Loading weights:  17%|▏| 25/146 [00:00<00:00, 2246.50it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:  17%|▏| 25/146 [00:00<00:00, 2226.28it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:  18%|▏| 26/146 [00:00<00:00, 2284.38it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:  18%|▏| 26/146 [00:00<00:00, 2265.31it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:  18%|▏| 27/146 [00:00<00:00, 2308.27it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:  18%|▏| 27/146 [00:00<00:00, 2288.87it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:  19%|▏| 28/146 [00:00<00:00, 2342.67it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:  19%|▏| 28/146 [00:00<00:00, 2288.80it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:  20%|▍ | 29/146 [00:00<00:00, 2285.34it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:  20%|▍ | 29/146 [00:00<00:00, 2254.58it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:  21%|▊   | 30/146 [00:00<00:00, 2288.47it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:  21%|▊   | 30/146 [00:00<00:00, 2270.79it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:  21%|▊   | 31/146 [00:00<00:00, 2270.24it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:  21%|▊   | 31/146 [00:00<00:00, 2253.63it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:  22%|█▎    | 32/146 [00:00<00:00, 2279.82it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:  22%|█▎    | 32/146 [00:00<00:00, 2263.22it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:  23%|▏| 33/146 [00:00<00:00, 2305.98it/s, Materializing param=model.layers.3.post_attention_layernorm
Loading weights:  23%|▏| 33/146 [00:00<00:00, 2270.32it/s, Materializing param=model.layers.3.post_attention_layernorm
Loading weights:  23%|▏| 34/146 [00:00<00:00, 2311.77it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  23%|▏| 34/146 [00:00<00:00, 2296.95it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  24%|▏| 35/146 [00:00<00:00, 2327.44it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  24%|▏| 35/146 [00:00<00:00, 2312.44it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  25%|▏| 36/146 [00:00<00:00, 2352.46it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  25%|▏| 36/146 [00:00<00:00, 2330.78it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  25%|▎| 37/146 [00:00<00:00, 2344.89it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  25%|▎| 37/146 [00:00<00:00, 2330.38it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  26%|▌ | 38/146 [00:00<00:00, 2361.13it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  26%|▌ | 38/146 [00:00<00:00, 2347.33it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  27%|█   | 39/146 [00:00<00:00, 2386.88it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  27%|█   | 39/146 [00:00<00:00, 2373.31it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  27%|█   | 40/146 [00:00<00:00, 2412.04it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  27%|█   | 40/146 [00:00<00:00, 2398.66it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  28%|█▋    | 41/146 [00:00<00:00, 2435.20it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  28%|█▋    | 41/146 [00:00<00:00, 2421.24it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  29%|▎| 42/146 [00:00<00:00, 2458.18it/s, Materializing param=model.layers.4.post_attention_layernorm
Loading weights:  29%|▎| 42/146 [00:00<00:00, 2443.52it/s, Materializing param=model.layers.4.post_attention_layernorm
Loading weights:  29%|▎| 43/146 [00:00<00:00, 2478.09it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  29%|▎| 43/146 [00:00<00:00, 2464.94it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  30%|▎| 44/146 [00:00<00:00, 2500.77it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  30%|▎| 44/146 [00:00<00:00, 2487.56it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  31%|▎| 45/146 [00:00<00:00, 2521.26it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  31%|▎| 45/146 [00:00<00:00, 2507.92it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  32%|▎| 46/146 [00:00<00:00, 2541.83it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  32%|▎| 46/146 [00:00<00:00, 2528.28it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  32%|▋ | 47/146 [00:00<00:00, 2560.09it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  32%|▋ | 47/146 [00:00<00:00, 2546.57it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  33%|█▎  | 48/146 [00:00<00:00, 2579.69it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  33%|█▎  | 48/146 [00:00<00:00, 2566.53it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  34%|█▎  | 49/146 [00:00<00:00, 2597.45it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  34%|█▎  | 49/146 [00:00<00:00, 2584.10it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  34%|██    | 50/146 [00:00<00:00, 2615.49it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  34%|██    | 50/146 [00:00<00:00, 2582.45it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  35%|▎| 51/146 [00:00<00:00, 2610.40it/s, Materializing param=model.layers.5.post_attention_layernorm
Loading weights:  35%|▎| 51/146 [00:00<00:00, 2597.03it/s, Materializing param=model.layers.5.post_attention_layernorm
Loading weights:  36%|▎| 52/146 [00:00<00:00, 2627.41it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  36%|▎| 52/146 [00:00<00:00, 2614.56it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  36%|▎| 53/146 [00:00<00:00, 2642.07it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  36%|▎| 53/146 [00:00<00:00, 2628.76it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  37%|▎| 54/146 [00:00<00:00, 2650.37it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  37%|▎| 54/146 [00:00<00:00, 2638.05it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  38%|▍| 55/146 [00:00<00:00, 2665.33it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  38%|▍| 55/146 [00:00<00:00, 2653.16it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  38%|▊ | 56/146 [00:00<00:00, 2681.87it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  38%|▊ | 56/146 [00:00<00:00, 2669.83it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  39%|█▌  | 57/146 [00:00<00:00, 2696.94it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  39%|█▌  | 57/146 [00:00<00:00, 2684.70it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  40%|█▌  | 58/146 [00:00<00:00, 2710.25it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  40%|█▌  | 58/146 [00:00<00:00, 2698.08it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  40%|██▍   | 59/146 [00:00<00:00, 2724.23it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  40%|██▍   | 59/146 [00:00<00:00, 2711.93it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  41%|▍| 60/146 [00:00<00:00, 2738.54it/s, Materializing param=model.layers.6.post_attention_layernorm
Loading weights:  41%|▍| 60/146 [00:00<00:00, 2726.79it/s, Materializing param=model.layers.6.post_attention_layernorm
Loading weights:  42%|▍| 61/146 [00:00<00:00, 2751.55it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  42%|▍| 61/146 [00:00<00:00, 2739.85it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  42%|▍| 62/146 [00:00<00:00, 2765.96it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  42%|▍| 62/146 [00:00<00:00, 2754.27it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  43%|▍| 63/146 [00:00<00:00, 2778.74it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  43%|▍| 63/146 [00:00<00:00, 2766.55it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  44%|▍| 64/146 [00:00<00:00, 2769.83it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  44%|▍| 64/146 [00:00<00:00, 2758.62it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  45%|▉ | 65/146 [00:00<00:00, 2781.77it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  45%|▉ | 65/146 [00:00<00:00, 2770.85it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  45%|█▊  | 66/146 [00:00<00:00, 2786.86it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  45%|█▊  | 66/146 [00:00<00:00, 2775.29it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  46%|█▊  | 67/146 [00:00<00:00, 2701.19it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  46%|█▊  | 67/146 [00:00<00:00, 2689.92it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  47%|██▊   | 68/146 [00:00<00:00, 2705.08it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  47%|██▊   | 68/146 [00:00<00:00, 2694.80it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  47%|▍| 69/146 [00:00<00:00, 2690.18it/s, Materializing param=model.layers.7.post_attention_layernorm
Loading weights:  47%|▍| 69/146 [00:00<00:00, 2679.87it/s, Materializing param=model.layers.7.post_attention_layernorm
Loading weights:  48%|▍| 70/146 [00:00<00:00, 2703.71it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  48%|▍| 70/146 [00:00<00:00, 2694.21it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  49%|▍| 71/146 [00:00<00:00, 2718.20it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  49%|▍| 71/146 [00:00<00:00, 2708.81it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  49%|▍| 72/146 [00:00<00:00, 2732.79it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  49%|▍| 72/146 [00:00<00:00, 2723.70it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  50%|▌| 73/146 [00:00<00:00, 2747.28it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  50%|▌| 73/146 [00:00<00:00, 2737.62it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  51%|█ | 74/146 [00:00<00:00, 2760.59it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  51%|█ | 74/146 [00:00<00:00, 2751.29it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  51%|██  | 75/146 [00:00<00:00, 2774.60it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  51%|██  | 75/146 [00:00<00:00, 2765.09it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  52%|██  | 76/146 [00:00<00:00, 2787.47it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  52%|██  | 76/146 [00:00<00:00, 2778.44it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  53%|███▏  | 77/146 [00:00<00:00, 2800.98it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  53%|███▏  | 77/146 [00:00<00:00, 2791.80it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  53%|▌| 78/146 [00:00<00:00, 2814.12it/s, Materializing param=model.layers.8.post_attention_layernorm
Loading weights:  53%|▌| 78/146 [00:00<00:00, 2804.74it/s, Materializing param=model.layers.8.post_attention_layernorm
Loading weights:  54%|▌| 79/146 [00:00<00:00, 2826.57it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  54%|▌| 79/146 [00:00<00:00, 2817.00it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  55%|▌| 80/146 [00:00<00:00, 2838.35it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  55%|▌| 80/146 [00:00<00:00, 2829.33it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  55%|▌| 81/146 [00:00<00:00, 2850.63it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  55%|▌| 81/146 [00:00<00:00, 2841.67it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  56%|▌| 82/146 [00:00<00:00, 2862.84it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  56%|▌| 82/146 [00:00<00:00, 2853.95it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  57%|█▏| 83/146 [00:00<00:00, 2875.11it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  57%|█▏| 83/146 [00:00<00:00, 2865.93it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  58%|██▎ | 84/146 [00:00<00:00, 2886.18it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  58%|██▎ | 84/146 [00:00<00:00, 2877.06it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  58%|██▎ | 85/146 [00:00<00:00, 2897.30it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  58%|██▎ | 85/146 [00:00<00:00, 2888.05it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  59%|███▌  | 86/146 [00:00<00:00, 2908.37it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  59%|███▌  | 86/146 [00:00<00:00, 2899.34it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  60%|▌| 87/146 [00:00<00:00, 2919.24it/s, Materializing param=model.layers.9.post_attention_layernorm
Loading weights:  60%|▌| 87/146 [00:00<00:00, 2909.62it/s, Materializing param=model.layers.9.post_attention_layernorm
Loading weights:  60%|▌| 88/146 [00:00<00:00, 2929.08it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  60%|▌| 88/146 [00:00<00:00, 2920.39it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  61%|▌| 89/146 [00:00<00:00, 2939.92it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  61%|▌| 89/146 [00:00<00:00, 2931.03it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  62%|▌| 90/146 [00:00<00:00, 2950.36it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  62%|▌| 90/146 [00:00<00:00, 2941.79it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  62%|▌| 91/146 [00:00<00:00, 2961.27it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  62%|▌| 91/146 [00:00<00:00, 2952.80it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  63%|▋| 92/146 [00:00<00:00, 2971.84it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  63%|▋| 92/146 [00:00<00:00, 2963.38it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  64%|█▉ | 93/146 [00:00<00:00, 2982.94it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  64%|█▉ | 93/146 [00:00<00:00, 2974.50it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  64%|█▉ | 94/146 [00:00<00:00, 2993.00it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  64%|█▉ | 94/146 [00:00<00:00, 2984.32it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  65%|███▎ | 95/146 [00:00<00:00, 3002.48it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  65%|███▎ | 95/146 [00:00<00:00, 2993.77it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  66%|▋| 96/146 [00:00<00:00, 3012.47it/s, Materializing param=model.layers.10.post_attention_layernor
Loading weights:  66%|▋| 96/146 [00:00<00:00, 3003.26it/s, Materializing param=model.layers.10.post_attention_layernor
Loading weights:  66%|▋| 97/146 [00:00<00:00, 3020.46it/s, Materializing param=model.layers.10.self_attn.k_proj.weight
Loading weights:  66%|▋| 97/146 [00:00<00:00, 3012.03it/s, Materializing param=model.layers.10.self_attn.k_proj.weight
Loading weights:  67%|▋| 98/146 [00:00<00:00, 3029.78it/s, Materializing param=model.layers.10.self_attn.o_proj.weight
Loading weights:  67%|▋| 98/146 [00:00<00:00, 3021.06it/s, Materializing param=model.layers.10.self_attn.o_proj.weight
Loading weights:  68%|▋| 99/146 [00:00<00:00, 3038.91it/s, Materializing param=model.layers.10.self_attn.q_proj.weight
Loading weights:  68%|▋| 99/146 [00:00<00:00, 3030.63it/s, Materializing param=model.layers.10.self_attn.q_proj.weight
Loading weights:  68%|▋| 100/146 [00:00<00:00, 3048.54it/s, Materializing param=model.layers.10.self_attn.v_proj.weigh
Loading weights:  68%|▋| 100/146 [00:00<00:00, 3039.48it/s, Materializing param=model.layers.10.self_attn.v_proj.weigh
Loading weights:  69%|▋| 101/146 [00:00<00:00, 3057.14it/s, Materializing param=model.layers.11.input_layernorm.weight
Loading weights:  69%|▋| 101/146 [00:00<00:00, 3049.06it/s, Materializing param=model.layers.11.input_layernorm.weight
Loading weights:  70%|█▍| 102/146 [00:00<00:00, 3066.91it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  70%|█▍| 102/146 [00:00<00:00, 3058.60it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  71%|█▍| 103/146 [00:00<00:00, 3076.10it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  71%|█▍| 103/146 [00:00<00:00, 3068.04it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  71%|██▊ | 104/146 [00:00<00:00, 3085.16it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  71%|██▊ | 104/146 [00:00<00:00, 3076.72it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  72%|▋| 105/146 [00:00<00:00, 3093.64it/s, Materializing param=model.layers.11.post_attention_layerno
Loading weights:  72%|▋| 105/146 [00:00<00:00, 3085.04it/s, Materializing param=model.layers.11.post_attention_layerno
Loading weights:  73%|▋| 106/146 [00:00<00:00, 3101.34it/s, Materializing param=model.layers.11.self_attn.k_proj.weigh
Loading weights:  73%|▋| 106/146 [00:00<00:00, 3093.14it/s, Materializing param=model.layers.11.self_attn.k_proj.weigh
Loading weights:  73%|▋| 107/146 [00:00<00:00, 3109.88it/s, Materializing param=model.layers.11.self_attn.o_proj.weigh
Loading weights:  73%|▋| 107/146 [00:00<00:00, 3101.46it/s, Materializing param=model.layers.11.self_attn.o_proj.weigh
Loading weights:  74%|▋| 108/146 [00:00<00:00, 3118.06it/s, Materializing param=model.layers.11.self_attn.q_proj.weigh
Loading weights:  74%|▋| 108/146 [00:00<00:00, 3109.71it/s, Materializing param=model.layers.11.self_attn.q_proj.weigh
Loading weights:  75%|▋| 109/146 [00:00<00:00, 3125.82it/s, Materializing param=model.layers.11.self_attn.v_proj.weigh
Loading weights:  75%|▋| 109/146 [00:00<00:00, 3117.55it/s, Materializing param=model.layers.11.self_attn.v_proj.weigh
Loading weights:  75%|▊| 110/146 [00:00<00:00, 3133.29it/s, Materializing param=model.layers.12.input_layernorm.weight
Loading weights:  75%|▊| 110/146 [00:00<00:00, 3125.52it/s, Materializing param=model.layers.12.input_layernorm.weight
Loading weights:  76%|█▌| 111/146 [00:00<00:00, 3142.06it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  76%|█▌| 111/146 [00:00<00:00, 3133.79it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  77%|█▌| 112/146 [00:00<00:00, 3149.47it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  77%|█▌| 112/146 [00:00<00:00, 3141.57it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  77%|███ | 113/146 [00:00<00:00, 3157.46it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  77%|███ | 113/146 [00:00<00:00, 3149.40it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  78%|▊| 114/146 [00:00<00:00, 3164.90it/s, Materializing param=model.layers.12.post_attention_layerno
Loading weights:  78%|▊| 114/146 [00:00<00:00, 3156.57it/s, Materializing param=model.layers.12.post_attention_layerno
Loading weights:  79%|▊| 115/146 [00:00<00:00, 3172.36it/s, Materializing param=model.layers.12.self_attn.k_proj.weigh
Loading weights:  79%|▊| 115/146 [00:00<00:00, 3164.83it/s, Materializing param=model.layers.12.self_attn.k_proj.weigh
Loading weights:  79%|▊| 116/146 [00:00<00:00, 3180.56it/s, Materializing param=model.layers.12.self_attn.o_proj.weigh
Loading weights:  79%|▊| 116/146 [00:00<00:00, 3172.70it/s, Materializing param=model.layers.12.self_attn.o_proj.weigh
Loading weights:  80%|▊| 117/146 [00:00<00:00, 3188.15it/s, Materializing param=model.layers.12.self_attn.q_proj.weigh
Loading weights:  80%|▊| 117/146 [00:00<00:00, 3179.99it/s, Materializing param=model.layers.12.self_attn.q_proj.weigh
Loading weights:  81%|▊| 118/146 [00:00<00:00, 3195.62it/s, Materializing param=model.layers.12.self_attn.v_proj.weigh
Loading weights:  81%|▊| 118/146 [00:00<00:00, 3188.02it/s, Materializing param=model.layers.12.self_attn.v_proj.weigh
Loading weights:  82%|▊| 119/146 [00:00<00:00, 3203.57it/s, Materializing param=model.layers.13.input_layernorm.weight
Loading weights:  82%|▊| 119/146 [00:00<00:00, 3195.94it/s, Materializing param=model.layers.13.input_layernorm.weight
Loading weights:  82%|█▋| 120/146 [00:00<00:00, 3211.09it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  82%|█▋| 120/146 [00:00<00:00, 3203.47it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  83%|█▋| 121/146 [00:00<00:00, 3218.57it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  83%|█▋| 121/146 [00:00<00:00, 3210.73it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  84%|███▎| 122/146 [00:00<00:00, 3225.59it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  84%|███▎| 122/146 [00:00<00:00, 3218.09it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  84%|▊| 123/146 [00:00<00:00, 3232.78it/s, Materializing param=model.layers.13.post_attention_layerno
Loading weights:  84%|▊| 123/146 [00:00<00:00, 3224.59it/s, Materializing param=model.layers.13.post_attention_layerno
Loading weights:  85%|▊| 124/146 [00:00<00:00, 3239.05it/s, Materializing param=model.layers.13.self_attn.k_proj.weigh
Loading weights:  85%|▊| 124/146 [00:00<00:00, 3231.12it/s, Materializing param=model.layers.13.self_attn.k_proj.weigh
Loading weights:  86%|▊| 125/146 [00:00<00:00, 3244.34it/s, Materializing param=model.layers.13.self_attn.o_proj.weigh
Loading weights:  86%|▊| 125/146 [00:00<00:00, 3236.71it/s, Materializing param=model.layers.13.self_attn.o_proj.weigh
Loading weights:  86%|▊| 126/146 [00:00<00:00, 3250.90it/s, Materializing param=model.layers.13.self_attn.q_proj.weigh
Loading weights:  86%|▊| 126/146 [00:00<00:00, 3242.90it/s, Materializing param=model.layers.13.self_attn.q_proj.weigh
Loading weights:  87%|▊| 127/146 [00:00<00:00, 3256.13it/s, Materializing param=model.layers.13.self_attn.v_proj.weigh
Loading weights:  87%|▊| 127/146 [00:00<00:00, 3248.23it/s, Materializing param=model.layers.13.self_attn.v_proj.weigh
Loading weights:  88%|▉| 128/146 [00:00<00:00, 3262.11it/s, Materializing param=model.layers.14.input_layernorm.weight
Loading weights:  88%|▉| 128/146 [00:00<00:00, 3254.26it/s, Materializing param=model.layers.14.input_layernorm.weight
Loading weights:  88%|█▊| 129/146 [00:00<00:00, 3268.17it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  88%|█▊| 129/146 [00:00<00:00, 3260.49it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  89%|█▊| 130/146 [00:00<00:00, 3274.08it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  89%|█▊| 130/146 [00:00<00:00, 3266.51it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  90%|███▌| 131/146 [00:00<00:00, 3280.07it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  90%|███▌| 131/146 [00:00<00:00, 3271.71it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  90%|▉| 132/146 [00:00<00:00, 3284.61it/s, Materializing param=model.layers.14.post_attention_layerno
Loading weights:  90%|▉| 132/146 [00:00<00:00, 3276.53it/s, Materializing param=model.layers.14.post_attention_layerno
Loading weights:  91%|▉| 133/146 [00:00<00:00, 3289.65it/s, Materializing param=model.layers.14.self_attn.k_proj.weigh
Loading weights:  91%|▉| 133/146 [00:00<00:00, 3282.10it/s, Materializing param=model.layers.14.self_attn.k_proj.weigh
Loading weights:  92%|▉| 134/146 [00:00<00:00, 3294.99it/s, Materializing param=model.layers.14.self_attn.o_proj.weigh
Loading weights:  92%|▉| 134/146 [00:00<00:00, 3287.73it/s, Materializing param=model.layers.14.self_attn.o_proj.weigh
Loading weights:  92%|▉| 135/146 [00:00<00:00, 3300.98it/s, Materializing param=model.layers.14.self_attn.q_proj.weigh
Loading weights:  92%|▉| 135/146 [00:00<00:00, 3293.59it/s, Materializing param=model.layers.14.self_attn.q_proj.weigh
Loading weights:  93%|▉| 136/146 [00:00<00:00, 3306.60it/s, Materializing param=model.layers.14.self_attn.v_proj.weigh
Loading weights:  93%|▉| 136/146 [00:00<00:00, 3299.51it/s, Materializing param=model.layers.14.self_attn.v_proj.weigh
Loading weights:  94%|▉| 137/146 [00:00<00:00, 3312.77it/s, Materializing param=model.layers.15.input_layernorm.weight
Loading weights:  94%|▉| 137/146 [00:00<00:00, 3305.32it/s, Materializing param=model.layers.15.input_layernorm.weight
Loading weights:  95%|█▉| 138/146 [00:00<00:00, 3318.39it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  95%|█▉| 138/146 [00:00<00:00, 3311.14it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  95%|█▉| 139/146 [00:00<00:00, 3324.35it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  95%|█▉| 139/146 [00:00<00:00, 3316.99it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  96%|███▊| 140/146 [00:00<00:00, 3329.81it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  96%|███▊| 140/146 [00:00<00:00, 3322.95it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  97%|▉| 141/146 [00:00<00:00, 3335.99it/s, Materializing param=model.layers.15.post_attention_layerno
Loading weights:  97%|▉| 141/146 [00:00<00:00, 3328.64it/s, Materializing param=model.layers.15.post_attention_layerno
Loading weights:  97%|▉| 142/146 [00:00<00:00, 3341.25it/s, Materializing param=model.layers.15.self_attn.k_proj.weigh
Loading weights:  97%|▉| 142/146 [00:00<00:00, 3334.14it/s, Materializing param=model.layers.15.self_attn.k_proj.weigh
Loading weights:  98%|▉| 143/146 [00:00<00:00, 3346.87it/s, Materializing param=model.layers.15.self_attn.o_proj.weigh
Loading weights:  98%|▉| 143/146 [00:00<00:00, 3339.73it/s, Materializing param=model.layers.15.self_attn.o_proj.weigh
Loading weights:  99%|▉| 144/146 [00:00<00:00, 3352.24it/s, Materializing param=model.layers.15.self_attn.q_proj.weigh
Loading weights:  99%|▉| 144/146 [00:00<00:00, 3345.20it/s, Materializing param=model.layers.15.self_attn.q_proj.weigh
Loading weights:  99%|▉| 145/146 [00:00<00:00, 3357.85it/s, Materializing param=model.layers.15.self_attn.v_proj.weigh
Loading weights:  99%|▉| 145/146 [00:00<00:00, 3350.73it/s, Materializing param=model.layers.15.self_attn.v_proj.weigh
Loading weights: 100%|█████████████████████| 146/146 [00:00<00:00, 3363.07it/s, Materializing param=model.norm.weight]
Loading weights: 100%|█████████████████████| 146/146 [00:00<00:00, 3355.99it/s, Materializing param=model.norm.weight]
Loading weights: 100%|█████████████████████| 146/146 [00:00<00:00, 3341.97it/s, Materializing param=model.norm.weight]
+[2026-02-10 15:44:22,256] [WARNING] [torchao.<module>:39] [PID:14756] Skipping import of cpp extensions due to incompatible torch version 2.9.1+cu128 for torchao version 0.13.0
+[2026-02-10 15:44:41,826] [WARNING] [accelerate.utils.dataclasses.__post_init__:1962] [PID:14756] sharding_strategy is deprecated in favor of reshard_after_forward. This will be removed in a future version of Accelerate.
+[2026-02-10 16:12:36,776] [WARNING] [py.warnings._showwarnmsg:110] [PID:14756] /root/miniconda3/envs/py3.11/lib/python3.11/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html .
   warnings.warn(