diff --git "a/debug.log" "b/debug.log"
--- "a/debug.log"
+++ "b/debug.log"
@@ -1,10 +1,7 @@
-
Loading dataset from disk:   0%|                                                                                             | 0/205 [00:00<?, ?it/s]
Loading dataset from disk: 100%|████████████████████████████████████████████████████████████████████████████████| 205/205 [00:00<00:00, 43768.51it/s]
-
Loading weights:   0%|                                                                                                       | 0/338 [00:00<?, ?it/s]
Loading weights:   0%|▏                                             | 1/338 [00:00<00:00, 8665.92it/s, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|▏                                             | 1/338 [00:00<00:00, 4644.85it/s, Materializing param=model.embed_tokens.weight]
Loading weights:   1%|▏                                 | 2/338 [00:00<00:00, 5080.93it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|▏                                 | 2/338 [00:00<00:00, 4230.26it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|▎                                   | 3/338 [00:00<00:00, 5031.15it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   1%|▎                                   | 3/338 [00:00<00:00, 4353.95it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   1%|▍                                   | 4/338 [00:00<00:00, 4792.12it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|▍                                   | 4/338 [00:00<00:00, 4405.78it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|▌                                     | 5/338 [00:00<00:00, 2653.62it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   1%|▌                                     | 5/338 [00:00<00:00, 2539.85it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   2%|▍                        | 6/338 [00:00<00:00, 2731.85it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▍                        | 6/338 [00:00<00:00, 2644.30it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▋                                  | 7/338 [00:00<00:00, 2946.62it/s, Materializing param=model.layers.0.self_attn.k_proj.bias]
Loading weights:   2%|▋                                  | 7/338 [00:00<00:00, 2871.97it/s, Materializing param=model.layers.0.self_attn.k_proj.bias]
Loading weights:   2%|▊                                | 8/338 [00:00<00:00, 3137.69it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|▊                                | 8/338 [00:00<00:00, 3060.42it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   3%|▉                                | 9/338 [00:00<00:00, 3277.94it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   3%|▉                                | 9/338 [00:00<00:00, 3196.07it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   3%|█                                 | 10/338 [00:00<00:00, 3428.68it/s, Materializing param=model.layers.0.self_attn.q_proj.bias]
Loading weights:   3%|█                                 | 10/338 [00:00<00:00, 3361.36it/s, Materializing param=model.layers.0.self_attn.q_proj.bias]
Loading weights:   3%|█                               | 11/338 [00:00<00:00, 3564.11it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|█                               | 11/338 [00:00<00:00, 3493.93it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   4%|█▏                                | 12/338 [00:00<00:00, 3493.56it/s, Materializing param=model.layers.0.self_attn.v_proj.bias]
Loading weights:   4%|█▏                                | 12/338 [00:00<00:00, 3407.00it/s, Materializing param=model.layers.0.self_attn.v_proj.bias]
Loading weights:   4%|█▏                              | 13/338 [00:00<00:00, 3584.64it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   4%|█▏                              | 13/338 [00:00<00:00, 3515.76it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   4%|█▎                               | 14/338 [00:00<00:00, 3243.68it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   4%|█▎                               | 14/338 [00:00<00:00, 3190.45it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   4%|█▌                                 | 15/338 [00:00<00:00, 3331.28it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   4%|█▌                                 | 15/338 [00:00<00:00, 3286.39it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   5%|█▋                                 | 16/338 [00:00<00:00, 3424.27it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   5%|█▋                                 | 16/338 [00:00<00:00, 3370.44it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   5%|█▊                                   | 17/338 [00:00<00:00, 3497.14it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   5%|█▊                                   | 17/338 [00:00<00:00, 3453.27it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   5%|█▎                      | 18/338 [00:00<00:00, 3345.93it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   5%|█▎                      | 18/338 [00:00<00:00, 3303.76it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   6%|█▉                                | 19/338 [00:00<00:00, 3357.00it/s, Materializing param=model.layers.1.self_attn.k_proj.bias]
Loading weights:   6%|█▉                                | 19/338 [00:00<00:00, 3311.93it/s, Materializing param=model.layers.1.self_attn.k_proj.bias]
Loading weights:   6%|█▉                              | 20/338 [00:00<00:00, 3413.61it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   6%|█▉                              | 20/338 [00:00<00:00, 3377.74it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   6%|█▉                              | 21/338 [00:00<00:00, 3430.19it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   6%|█▉                              | 21/338 [00:00<00:00, 3395.54it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   7%|██▏                               | 22/338 [00:00<00:00, 3493.00it/s, Materializing param=model.layers.1.self_attn.q_proj.bias]
Loading weights:   7%|██▏                               | 22/338 [00:00<00:00, 3460.52it/s, Materializing param=model.layers.1.self_attn.q_proj.bias]
Loading weights:   7%|██▏                             | 23/338 [00:00<00:00, 3455.44it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   7%|██▏                             | 23/338 [00:00<00:00, 3416.16it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   7%|██▍                               | 24/338 [00:00<00:00, 3516.01it/s, Materializing param=model.layers.1.self_attn.v_proj.bias]
Loading weights:   7%|██▍                               | 24/338 [00:00<00:00, 3430.92it/s, Materializing param=model.layers.1.self_attn.v_proj.bias]
Loading weights:   7%|██▎                             | 25/338 [00:00<00:00, 3458.71it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   7%|██▎                             | 25/338 [00:00<00:00, 3427.05it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   8%|██▌                              | 26/338 [00:00<00:00, 3429.20it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   8%|██▌                              | 26/338 [00:00<00:00, 3399.16it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   8%|██▊                                | 27/338 [00:00<00:00, 3480.75it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   8%|██▊                                | 27/338 [00:00<00:00, 3453.68it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   8%|██▉                                | 28/338 [00:00<00:00, 3531.51it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   8%|██▉                                | 28/338 [00:00<00:00, 3504.95it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   9%|███▏                                 | 29/338 [00:00<00:00, 3589.00it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   9%|███▏                                 | 29/338 [00:00<00:00, 3562.93it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   9%|██▏                     | 30/338 [00:00<00:00, 3476.04it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   9%|██▏                     | 30/338 [00:00<00:00, 3331.72it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   9%|███                               | 31/338 [00:00<00:00, 3399.13it/s, Materializing param=model.layers.2.self_attn.k_proj.bias]
Loading weights:   9%|███                               | 31/338 [00:00<00:00, 3376.62it/s, Materializing param=model.layers.2.self_attn.k_proj.bias]
Loading weights:   9%|███                             | 32/338 [00:00<00:00, 3443.07it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   9%|███                             | 32/338 [00:00<00:00, 3419.56it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:  10%|███                             | 33/338 [00:00<00:00, 3492.17it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:  10%|███                             | 33/338 [00:00<00:00, 3470.63it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:  10%|███▍                              | 34/338 [00:00<00:00, 3533.97it/s, Materializing param=model.layers.2.self_attn.q_proj.bias]
Loading weights:  10%|███▍                              | 34/338 [00:00<00:00, 3508.50it/s, Materializing param=model.layers.2.self_attn.q_proj.bias]
Loading weights:  10%|███▎                            | 35/338 [00:00<00:00, 3533.28it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:  10%|███▎                            | 35/338 [00:00<00:00, 3510.21it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:  11%|███▌                              | 36/338 [00:00<00:00, 3568.86it/s, Materializing param=model.layers.2.self_attn.v_proj.bias]
Loading weights:  11%|███▌                              | 36/338 [00:00<00:00, 3547.31it/s, Materializing param=model.layers.2.self_attn.v_proj.bias]
Loading weights:  11%|███▌                            | 37/338 [00:00<00:00, 3560.45it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:  11%|███▌                            | 37/338 [00:00<00:00, 3538.85it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:  11%|███▋                             | 38/338 [00:00<00:00, 3534.16it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:  11%|███▋                             | 38/338 [00:00<00:00, 3510.03it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:  12%|████                               | 39/338 [00:00<00:00, 3532.62it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:  12%|████                               | 39/338 [00:00<00:00, 3511.46it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:  12%|████▏                              | 40/338 [00:00<00:00, 3571.82it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:  12%|████▏                              | 40/338 [00:00<00:00, 3553.22it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:  12%|████▍                                | 41/338 [00:00<00:00, 3576.67it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:  12%|████▍                                | 41/338 [00:00<00:00, 3343.24it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:  12%|██▉                     | 42/338 [00:00<00:00, 3392.60it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:  12%|██▉                     | 42/338 [00:00<00:00, 3375.96it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:  13%|████▎                             | 43/338 [00:00<00:00, 3342.51it/s, Materializing param=model.layers.3.self_attn.k_proj.bias]
Loading weights:  13%|████▎                             | 43/338 [00:00<00:00, 3324.52it/s, Materializing param=model.layers.3.self_attn.k_proj.bias]
Loading weights:  13%|████▏                           | 44/338 [00:00<00:00, 3376.13it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  13%|████▏                           | 44/338 [00:00<00:00, 3361.25it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  13%|████▎                           | 45/338 [00:00<00:00, 3311.35it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  13%|████▎                           | 45/338 [00:00<00:00, 3293.61it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  14%|████▋                             | 46/338 [00:00<00:00, 3333.82it/s, Materializing param=model.layers.3.self_attn.q_proj.bias]
Loading weights:  14%|████▋                             | 46/338 [00:00<00:00, 3318.91it/s, Materializing param=model.layers.3.self_attn.q_proj.bias]
Loading weights:  14%|████▍                           | 47/338 [00:00<00:00, 3279.36it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  14%|████▍                           | 47/338 [00:00<00:00, 3263.67it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  14%|████▊                             | 48/338 [00:00<00:00, 3307.87it/s, Materializing param=model.layers.3.self_attn.v_proj.bias]
Loading weights:  14%|████▊                             | 48/338 [00:00<00:00, 3293.96it/s, Materializing param=model.layers.3.self_attn.v_proj.bias]
Loading weights:  14%|████▋                           | 49/338 [00:00<00:00, 3219.77it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  14%|████▋                           | 49/338 [00:00<00:00, 3203.86it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  15%|████▉                            | 50/338 [00:00<00:00, 3245.06it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  15%|████▉                            | 50/338 [00:00<00:00, 3232.31it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  15%|█████▎                             | 51/338 [00:00<00:00, 3276.90it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  15%|█████▎                             | 51/338 [00:00<00:00, 3264.80it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  15%|█████▍                             | 52/338 [00:00<00:00, 3302.55it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  15%|█████▍                             | 52/338 [00:00<00:00, 3229.40it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  16%|█████▊                               | 53/338 [00:00<00:00, 3268.71it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  16%|█████▊                               | 53/338 [00:00<00:00, 3256.92it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  16%|███▊                    | 54/338 [00:00<00:00, 3294.72it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  16%|███▊                    | 54/338 [00:00<00:00, 3282.50it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  16%|█████▌                            | 55/338 [00:00<00:00, 3324.73it/s, Materializing param=model.layers.4.self_attn.k_proj.bias]
Loading weights:  16%|█████▌                            | 55/338 [00:00<00:00, 3313.32it/s, Materializing param=model.layers.4.self_attn.k_proj.bias]
Loading weights:  17%|█████▎                          | 56/338 [00:00<00:00, 3256.31it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  17%|█████▎                          | 56/338 [00:00<00:00, 3242.11it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  17%|█████▍                          | 57/338 [00:00<00:00, 3281.16it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  17%|█████▍                          | 57/338 [00:00<00:00, 3270.12it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  17%|█████▊                            | 58/338 [00:00<00:00, 3251.62it/s, Materializing param=model.layers.4.self_attn.q_proj.bias]
Loading weights:  17%|█████▊                            | 58/338 [00:00<00:00, 3124.57it/s, Materializing param=model.layers.4.self_attn.q_proj.bias]
Loading weights:  17%|█████▌                          | 59/338 [00:00<00:00, 3154.50it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  17%|█████▌                          | 59/338 [00:00<00:00, 3001.13it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  18%|██████                            | 60/338 [00:00<00:00, 3031.70it/s, Materializing param=model.layers.4.self_attn.v_proj.bias]
Loading weights:  18%|██████                            | 60/338 [00:00<00:00, 2962.22it/s, Materializing param=model.layers.4.self_attn.v_proj.bias]
Loading weights:  18%|█████▊                          | 61/338 [00:00<00:00, 2993.76it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  18%|█████▊                          | 61/338 [00:00<00:00, 2984.54it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  18%|██████                           | 62/338 [00:00<00:00, 3017.48it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  18%|██████                           | 62/338 [00:00<00:00, 3008.58it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  19%|██████▌                            | 63/338 [00:00<00:00, 3041.45it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  19%|██████▌                            | 63/338 [00:00<00:00, 3031.78it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  19%|██████▋                            | 64/338 [00:00<00:00, 3062.93it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  19%|██████▋                            | 64/338 [00:00<00:00, 2983.74it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  19%|███████                              | 65/338 [00:00<00:00, 2990.02it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  19%|███████                              | 65/338 [00:00<00:00, 2979.36it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  20%|████▋                   | 66/338 [00:00<00:00, 3010.69it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  20%|████▋                   | 66/338 [00:00<00:00, 3002.37it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  20%|██████▋                           | 67/338 [00:00<00:00, 3035.51it/s, Materializing param=model.layers.5.self_attn.k_proj.bias]
Loading weights:  20%|██████▋                           | 67/338 [00:00<00:00, 3027.69it/s, Materializing param=model.layers.5.self_attn.k_proj.bias]
Loading weights:  20%|██████▍                         | 68/338 [00:00<00:00, 3029.92it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  20%|██████▍                         | 68/338 [00:00<00:00, 3020.58it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  20%|██████▌                         | 69/338 [00:00<00:00, 3049.54it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  20%|██████▌                         | 69/338 [00:00<00:00, 3039.77it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  21%|███████                           | 70/338 [00:00<00:00, 3018.88it/s, Materializing param=model.layers.5.self_attn.q_proj.bias]
Loading weights:  21%|███████                           | 70/338 [00:00<00:00, 3009.78it/s, Materializing param=model.layers.5.self_attn.q_proj.bias]
Loading weights:  21%|██████▋                         | 71/338 [00:00<00:00, 3023.09it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  21%|██████▋                         | 71/338 [00:00<00:00, 3014.46it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  21%|███████▏                          | 72/338 [00:00<00:00, 3042.35it/s, Materializing param=model.layers.5.self_attn.v_proj.bias]
Loading weights:  21%|███████▏                          | 72/338 [00:00<00:00, 3011.95it/s, Materializing param=model.layers.5.self_attn.v_proj.bias]
Loading weights:  22%|██████▉                         | 73/338 [00:00<00:00, 3037.21it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  22%|██████▉                         | 73/338 [00:00<00:00, 3029.46it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  22%|███████▏                         | 74/338 [00:00<00:00, 3059.63it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  22%|███████▏                         | 74/338 [00:00<00:00, 3052.68it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  22%|███████▊                           | 75/338 [00:00<00:00, 3010.38it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  22%|███████▊                           | 75/338 [00:00<00:00, 2959.71it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  22%|███████▊                           | 76/338 [00:00<00:00, 2985.44it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  22%|███████▊                           | 76/338 [00:00<00:00, 2978.32it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  23%|████████▍                            | 77/338 [00:00<00:00, 2992.21it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  23%|████████▍                            | 77/338 [00:00<00:00, 2983.78it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  23%|█████▌                  | 78/338 [00:00<00:00, 3010.79it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  23%|█████▌                  | 78/338 [00:00<00:00, 3003.69it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  23%|███████▉                          | 79/338 [00:00<00:00, 3029.49it/s, Materializing param=model.layers.6.self_attn.k_proj.bias]
Loading weights:  23%|███████▉                          | 79/338 [00:00<00:00, 3022.33it/s, Materializing param=model.layers.6.self_attn.k_proj.bias]
Loading weights:  24%|███████▌                        | 80/338 [00:00<00:00, 3049.60it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  24%|███████▌                        | 80/338 [00:00<00:00, 3043.07it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  24%|███████▋                        | 81/338 [00:00<00:00, 3051.96it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  24%|███████▋                        | 81/338 [00:00<00:00, 3044.94it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  24%|████████▏                         | 82/338 [00:00<00:00, 3025.98it/s, Materializing param=model.layers.6.self_attn.q_proj.bias]
Loading weights:  24%|████████▏                         | 82/338 [00:00<00:00, 2926.72it/s, Materializing param=model.layers.6.self_attn.q_proj.bias]
Loading weights:  25%|███████▊                        | 83/338 [00:00<00:00, 2946.06it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  25%|███████▊                        | 83/338 [00:00<00:00, 2939.32it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  25%|████████▍                         | 84/338 [00:00<00:00, 2905.41it/s, Materializing param=model.layers.6.self_attn.v_proj.bias]
Loading weights:  25%|████████▍                         | 84/338 [00:00<00:00, 2866.13it/s, Materializing param=model.layers.6.self_attn.v_proj.bias]
Loading weights:  25%|████████                        | 85/338 [00:00<00:00, 2888.22it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  25%|████████                        | 85/338 [00:00<00:00, 2881.70it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  25%|████████▍                        | 86/338 [00:00<00:00, 2889.66it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  25%|████████▍                        | 86/338 [00:00<00:00, 2883.05it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  26%|█████████                          | 87/338 [00:00<00:00, 2890.17it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  26%|█████████                          | 87/338 [00:00<00:00, 2883.53it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  26%|█████████                          | 88/338 [00:00<00:00, 2880.59it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  26%|█████████                          | 88/338 [00:00<00:00, 2873.95it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  26%|█████████▋                           | 89/338 [00:00<00:00, 2896.37it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  26%|█████████▋                           | 89/338 [00:00<00:00, 2861.38it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  27%|██████▍                 | 90/338 [00:00<00:00, 2838.51it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  27%|██████▍                 | 90/338 [00:00<00:00, 2831.46it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  27%|█████████▏                        | 91/338 [00:00<00:00, 2841.61it/s, Materializing param=model.layers.7.self_attn.k_proj.bias]
Loading weights:  27%|█████████▏                        | 91/338 [00:00<00:00, 2835.78it/s, Materializing param=model.layers.7.self_attn.k_proj.bias]
Loading weights:  27%|████████▋                       | 92/338 [00:00<00:00, 2856.99it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  27%|████████▋                       | 92/338 [00:00<00:00, 2851.60it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  28%|████████▊                       | 93/338 [00:00<00:00, 2838.67it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  28%|████████▊                       | 93/338 [00:00<00:00, 2832.63it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  28%|█████████▍                        | 94/338 [00:00<00:00, 2853.23it/s, Materializing param=model.layers.7.self_attn.q_proj.bias]
Loading weights:  28%|█████████▍                        | 94/338 [00:00<00:00, 2848.03it/s, Materializing param=model.layers.7.self_attn.q_proj.bias]
Loading weights:  28%|████████▉                       | 95/338 [00:00<00:00, 2870.35it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  28%|████████▉                       | 95/338 [00:00<00:00, 2865.35it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  28%|█████████▋                        | 96/338 [00:00<00:00, 2886.40it/s, Materializing param=model.layers.7.self_attn.v_proj.bias]
Loading weights:  28%|█████████▋                        | 96/338 [00:00<00:00, 2881.22it/s, Materializing param=model.layers.7.self_attn.v_proj.bias]
Loading weights:  29%|█████████▏                      | 97/338 [00:00<00:00, 2856.57it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  29%|█████████▏                      | 97/338 [00:00<00:00, 2850.87it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  29%|█████████▌                       | 98/338 [00:00<00:00, 2872.01it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  29%|█████████▌                       | 98/338 [00:00<00:00, 2867.10it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  29%|██████████▎                        | 99/338 [00:00<00:00, 2887.07it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  29%|██████████▎                        | 99/338 [00:00<00:00, 2881.98it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  30%|██████████                        | 100/338 [00:00<00:00, 2891.77it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  30%|██████████                        | 100/338 [00:00<00:00, 2886.45it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  30%|██████████▊                         | 101/338 [00:00<00:00, 2896.88it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  30%|██████████▊                         | 101/338 [00:00<00:00, 2858.54it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  30%|██████▉                | 102/338 [00:00<00:00, 2877.56it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  30%|██████▉                | 102/338 [00:00<00:00, 2842.78it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  30%|██████████                       | 103/338 [00:00<00:00, 2861.36it/s, Materializing param=model.layers.8.self_attn.k_proj.bias]
Loading weights:  30%|██████████                       | 103/338 [00:00<00:00, 2856.55it/s, Materializing param=model.layers.8.self_attn.k_proj.bias]
Loading weights:  31%|█████████▌                     | 104/338 [00:00<00:00, 2875.37it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  31%|█████████▌                     | 104/338 [00:00<00:00, 2870.58it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  31%|█████████▋                     | 105/338 [00:00<00:00, 2890.90it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  31%|█████████▋                     | 105/338 [00:00<00:00, 2886.16it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  31%|██████████▎                      | 106/338 [00:00<00:00, 2904.87it/s, Materializing param=model.layers.8.self_attn.q_proj.bias]
Loading weights:  31%|██████████▎                      | 106/338 [00:00<00:00, 2900.15it/s, Materializing param=model.layers.8.self_attn.q_proj.bias]
Loading weights:  32%|█████████▊                     | 107/338 [00:00<00:00, 2920.06it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  32%|█████████▊                     | 107/338 [00:00<00:00, 2837.23it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  32%|██████████▌                      | 108/338 [00:00<00:00, 2854.47it/s, Materializing param=model.layers.8.self_attn.v_proj.bias]
Loading weights:  32%|██████████▌                      | 108/338 [00:00<00:00, 2849.64it/s, Materializing param=model.layers.8.self_attn.v_proj.bias]
Loading weights:  32%|█████████▉                     | 109/338 [00:00<00:00, 2827.31it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  32%|█████████▉                     | 109/338 [00:00<00:00, 2822.74it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  33%|██���███████▍                     | 110/338 [00:00<00:00, 2841.46it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  33%|██████████▍                     | 110/338 [00:00<00:00, 2836.85it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  33%|███████████▏                      | 111/338 [00:00<00:00, 2848.12it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  33%|███████████▏                      | 111/338 [00:00<00:00, 2843.42it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  33%|███████████▎                      | 112/338 [00:00<00:00, 2860.98it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  33%|███████████▎                      | 112/338 [00:00<00:00, 2856.63it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  33%|████████████                        | 113/338 [00:00<00:00, 2863.13it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  33%|████████████                        | 113/338 [00:00<00:00, 2833.14it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  34%|███████▊               | 114/338 [00:00<00:00, 2849.66it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  34%|███████▊               | 114/338 [00:00<00:00, 2845.27it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  34%|███████████▏                     | 115/338 [00:00<00:00, 2855.58it/s, Materializing param=model.layers.9.self_attn.k_proj.bias]
Loading weights:  34%|███████████▏                     | 115/338 [00:00<00:00, 2851.06it/s, Materializing param=model.layers.9.self_attn.k_proj.bias]
Loading weights:  34%|██████████▋                    | 116/338 [00:00<00:00, 2856.89it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  34%|██████████▋                    | 116/338 [00:00<00:00, 2852.41it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  35%|██████████▋                    | 117/338 [00:00<00:00, 2859.77it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  35%|██████████▋                    | 117/338 [00:00<00:00, 2849.52it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  35%|███████████▌                     | 118/338 [00:00<00:00, 2833.02it/s, Materializing param=model.layers.9.self_attn.q_proj.bias]
Loading weights:  35%|███████████▌                     | 118/338 [00:00<00:00, 2827.88it/s, Materializing param=model.layers.9.self_attn.q_proj.bias]
Loading weights:  35%|██████████▉                    | 119/338 [00:00<00:00, 2845.33it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  35%|██████████▉                    | 119/338 [00:00<00:00, 2841.48it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  36%|███████████▋                     | 120/338 [00:00<00:00, 2841.41it/s, Materializing param=model.layers.9.self_attn.v_proj.bias]
Loading weights:  36%|███████████▋                     | 120/338 [00:00<00:00, 2777.04it/s, Materializing param=model.layers.9.self_attn.v_proj.bias]
Loading weights:  36%|███████████                    | 121/338 [00:00<00:00, 2791.45it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  36%|███████████                    | 121/338 [00:00<00:00, 2787.25it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  36%|███████████▏                   | 122/338 [00:00<00:00, 2804.20it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  36%|███████████▏                   | 122/338 [00:00<00:00, 2800.53it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  36%|████████████                     | 123/338 [00:00<00:00, 2814.48it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  36%|████████████                     | 123/338 [00:00<00:00, 2794.05it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  37%|████████████                     | 124/338 [00:00<00:00, 2809.49it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  37%|████████████                     | 124/338 [00:00<00:00, 2805.62it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  37%|████████████▉                      | 125/338 [00:00<00:00, 2758.21it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  37%|████████████▉                      | 125/338 [00:00<00:00, 2741.79it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  37%|████████▏             | 126/338 [00:00<00:00, 2755.18it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  37%|████████▏             | 126/338 [00:00<00:00, 2751.06it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  38%|████████████                    | 127/338 [00:00<00:00, 2763.46it/s, Materializing param=model.layers.10.self_attn.k_proj.bias]
Loading weights:  38%|████████████                    | 127/338 [00:00<00:00, 2759.62it/s, Materializing param=model.layers.10.self_attn.k_proj.bias]
Loading weights:  38%|███████████▎                  | 128/338 [00:00<00:00, 2774.83it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  38%|███████████▎                  | 128/338 [00:00<00:00, 2763.44it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  38%|███████████▍                  | 129/338 [00:00<00:00, 2778.68it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  38%|███████████▍                  | 129/338 [00:00<00:00, 2775.15it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  38%|████████████▎                   | 130/338 [00:00<00:00, 2789.56it/s, Materializing param=model.layers.10.self_attn.q_proj.bias]
Loading weights:  38%|████████████▎                   | 130/338 [00:00<00:00, 2786.00it/s, Materializing param=model.layers.10.self_attn.q_proj.bias]
Loading weights:  39%|███████████▋                  | 131/338 [00:00<00:00, 2782.13it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  39%|███████████▋                  | 131/338 [00:00<00:00, 2777.91it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  39%|████████████▍                   | 132/338 [00:00<00:00, 2792.42it/s, Materializing param=model.layers.10.self_attn.v_proj.bias]
Loading weights:  39%|████████████▍                   | 132/338 [00:00<00:00, 2788.84it/s, Materializing param=model.layers.10.self_attn.v_proj.bias]
Loading weights:  39%|███████████▊                  | 133/338 [00:00<00:00, 2804.40it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  39%|███████████▊                  | 133/338 [00:00<00:00, 2801.03it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  40%|████████████▎                  | 134/338 [00:00<00:00, 2815.84it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights:  40%|████████████▎                  | 134/338 [00:00<00:00, 2812.24it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights:  40%|█████████████▏                   | 135/338 [00:00<00:00, 2825.88it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  40%|█████████████▏                   | 135/338 [00:00<00:00, 2822.28it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  40%|█████████████▎                   | 136/338 [00:00<00:00, 2821.00it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  40%|█████████████▎                   | 136/338 [00:00<00:00, 2817.12it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  41%|██████████████▏                    | 137/338 [00:00<00:00, 2772.67it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  41%|██████████████▏                    | 137/338 [00:00<00:00, 2768.22it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  41%|████████▉             | 138/338 [00:00<00:00, 2772.20it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  41%|████████▉             | 138/338 [00:00<00:00, 2768.17it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  41%|█████████████▏                  | 139/338 [00:00<00:00, 2781.94it/s, Materializing param=model.layers.11.self_attn.k_proj.bias]
Loading weights:  41%|█████████████▏                  | 139/338 [00:00<00:00, 2778.55it/s, Materializing param=model.layers.11.self_attn.k_proj.bias]
Loading weights:  41%|████████████▍                 | 140/338 [00:00<00:00, 2793.26it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  41%|████████████▍                 | 140/338 [00:00<00:00, 2790.01it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  42%|████████████▌                 | 141/338 [00:00<00:00, 2803.45it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  42%|████████████▌                 | 141/338 [00:00<00:00, 2799.92it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  42%|█████████████▍                  | 142/338 [00:00<00:00, 2813.91it/s, Materializing param=model.layers.11.self_attn.q_proj.bias]
Loading weights:  42%|█████████████▍                  | 142/338 [00:00<00:00, 2810.45it/s, Materializing param=model.layers.11.self_attn.q_proj.bias]
Loading weights:  42%|████████████▋                 | 143/338 [00:00<00:00, 2825.19it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  42%|████████████▋                 | 143/338 [00:00<00:00, 2822.03it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  43%|█████████████▋                  | 144/338 [00:00<00:00, 2807.54it/s, Materializing param=model.layers.11.self_attn.v_proj.bias]
Loading weights:  43%|█████████████▋                  | 144/338 [00:00<00:00, 2803.68it/s, Materializing param=model.layers.11.self_attn.v_proj.bias]
Loading weights:  43%|████████████▊                 | 145/338 [00:00<00:00, 2817.71it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  43%|████████████▊                 | 145/338 [00:00<00:00, 2814.64it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  43%|█████████████▍                 | 146/338 [00:00<00:00, 2827.26it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights:  43%|█████████████▍                 | 146/338 [00:00<00:00, 2812.60it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights:  43%|██████████████▎                  | 147/338 [00:00<00:00, 2824.86it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  43%|██████████████▎                  | 147/338 [00:00<00:00, 2821.58it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  44%|██████████████▍                  | 148/338 [00:00<00:00, 2826.05it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  44%|██████████████▍                  | 148/338 [00:00<00:00, 2810.22it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  44%|███████████████▍                   | 149/338 [00:00<00:00, 2797.12it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  44%|███████████████▍                   | 149/338 [00:00<00:00, 2793.49it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  44%|█████████▊            | 150/338 [00:00<00:00, 2800.47it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  44%|█████████▊            | 150/338 [00:00<00:00, 2796.92it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  45%|██████████████▎                 | 151/338 [00:00<00:00, 2809.80it/s, Materializing param=model.layers.12.self_attn.k_proj.bias]
Loading weights:  45%|██████████████▎                 | 151/338 [00:00<00:00, 2806.56it/s, Materializing param=model.layers.12.self_attn.k_proj.bias]
Loading weights:  45%|█████████████▍                | 152/338 [00:00<00:00, 2814.56it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  45%|█████████████▍                | 152/338 [00:00<00:00, 2811.06it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  45%|█████████████▌                | 153/338 [00:00<00:00, 2823.85it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  45%|█████████████▌                | 153/338 [00:00<00:00, 2808.35it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  46%|██████████████▌                 | 154/338 [00:00<00:00, 2820.61it/s, Materializing param=model.layers.12.self_attn.q_proj.bias]
Loading weights:  46%|██████████████▌                 | 154/338 [00:00<00:00, 2817.23it/s, Materializing param=model.layers.12.self_attn.q_proj.bias]
Loading weights:  46%|█████████████▊                | 155/338 [00:00<00:00, 2829.52it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  46%|█████████████▊                | 155/338 [00:00<00:00, 2826.31it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  46%|██████████████▊                 | 156/338 [00:00<00:00, 2838.88it/s, Materializing param=model.layers.12.self_attn.v_proj.bias]
Loading weights:  46%|██████████████▊                 | 156/338 [00:00<00:00, 2835.88it/s, Materializing param=model.layers.12.self_attn.v_proj.bias]
Loading weights:  46%|█████████████▉                | 157/338 [00:00<00:00, 2849.28it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  46%|█████████████▉                | 157/338 [00:00<00:00, 2846.30it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  47%|██████████████▍                | 158/338 [00:00<00:00, 2858.83it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights:  47%|██████████████▍                | 158/338 [00:00<00:00, 2855.85it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights:  47%|███████████████▌                 | 159/338 [00:00<00:00, 2836.28it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  47%|███████████████▌                 | 159/338 [00:00<00:00, 2821.99it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  47%|███████████████▌                 | 160/338 [00:00<00:00, 2831.13it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  47%|███████████████▌                 | 160/338 [00:00<00:00, 2827.87it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  48%|████████████████▋                  | 161/338 [00:00<00:00, 2840.90it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  48%|████████████████▋                  | 161/338 [00:00<00:00, 2837.37it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  48%|██████████▌           | 162/338 [00:00<00:00, 2849.28it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  48%|██████████▌           | 162/338 [00:00<00:00, 2846.25it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  48%|███████████████▍                | 163/338 [00:00<00:00, 2859.32it/s, Materializing param=model.layers.13.self_attn.k_proj.bias]
Loading weights:  48%|███████████████▍                | 163/338 [00:00<00:00, 2856.50it/s, Materializing param=model.layers.13.self_attn.k_proj.bias]
Loading weights:  49%|██████████████▌               | 164/338 [00:00<00:00, 2858.77it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  49%|██████████████▌               | 164/338 [00:00<00:00, 2855.46it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  49%|██████████████▋               | 165/338 [00:00<00:00, 2867.47it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  49%|██████████████▋               | 165/338 [00:00<00:00, 2864.51it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  49%|███████████████▋                | 166/338 [00:00<00:00, 2877.01it/s, Materializing param=model.layers.13.self_attn.q_proj.bias]
Loading weights:  49%|███████████████▋                | 166/338 [00:00<00:00, 2873.95it/s, Materializing param=model.layers.13.self_attn.q_proj.bias]
Loading weights:  49%|██████████████▊               | 167/338 [00:00<00:00, 2873.71it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  49%|██████████████▊               | 167/338 [00:00<00:00, 2870.47it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  50%|██████████████���▉                | 168/338 [00:00<00:00, 2881.96it/s, Materializing param=model.layers.13.self_attn.v_proj.bias]
Loading weights:  50%|███████████████▉                | 168/338 [00:00<00:00, 2879.00it/s, Materializing param=model.layers.13.self_attn.v_proj.bias]
Loading weights:  50%|███████████████               | 169/338 [00:00<00:00, 2890.84it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  50%|███████████████               | 169/338 [00:00<00:00, 2887.83it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  50%|███████████████▌               | 170/338 [00:00<00:00, 2886.43it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights:  50%|███████████████▌               | 170/338 [00:00<00:00, 2882.94it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights:  51%|████████████████▋                | 171/338 [00:00<00:00, 2894.20it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  51%|████████████████▋                | 171/338 [00:00<00:00, 2890.97it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  51%|████████████████▊                | 172/338 [00:00<00:00, 2889.55it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  51%|████████████████▊                | 172/338 [00:00<00:00, 2886.55it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  51%|█████████████████▉                 | 173/338 [00:00<00:00, 2888.36it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  51%|█████████████████▉                 | 173/338 [00:00<00:00, 2875.57it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  51%|███████████▎          | 174/338 [00:00<00:00, 2875.09it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  51%|███████████▎          | 174/338 [00:00<00:00, 2872.00it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  52%|████████████████▌               | 175/338 [00:00<00:00, 2883.84it/s, Materializing param=model.layers.14.self_attn.k_proj.bias]
Loading weights:  52%|████████████████▌               | 175/338 [00:00<00:00, 2880.82it/s, Materializing param=model.layers.14.self_attn.k_proj.bias]
Loading weights:  52%|███████████████▌              | 176/338 [00:00<00:00, 2891.97it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  52%|███████████████▌              | 176/338 [00:00<00:00, 2889.09it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  52%|███████████████▋              | 177/338 [00:00<00:00, 2898.05it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  52%|███████████████▋              | 177/338 [00:00<00:00, 2894.88it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  53%|████████████████▊               | 178/338 [00:00<00:00, 2878.12it/s, Materializing param=model.layers.14.self_attn.q_proj.bias]
Loading weights:  53%|████████████████▊               | 178/338 [00:00<00:00, 2874.69it/s, Materializing param=model.layers.14.self_attn.q_proj.bias]
Loading weights:  53%|███████████████▉              | 179/338 [00:00<00:00, 2885.69it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  53%|███████████████▉              | 179/338 [00:00<00:00, 2882.85it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  53%|█████████████████               | 180/338 [00:00<00:00, 2894.55it/s, Materializing param=model.layers.14.self_attn.v_proj.bias]
Loading weights:  53%|█████████████████               | 180/338 [00:00<00:00, 2888.62it/s, Materializing param=model.layers.14.self_attn.v_proj.bias]
Loading weights:  54%|████████████████              | 181/338 [00:00<00:00, 2900.08it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  54%|████████████████              | 181/338 [00:00<00:00, 2897.37it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  54%|████████████████▋              | 182/338 [00:00<00:00, 2907.42it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights:  54%|████████████████▋              | 182/338 [00:00<00:00, 2904.47it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights:  54%|█████████████████▊               | 183/338 [00:00<00:00, 2916.05it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  54%|█████████████████▊               | 183/338 [00:00<00:00, 2913.47it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  54%|█████████████████▉               | 184/338 [00:00<00:00, 2925.76it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  54%|█████████████████▉               | 184/338 [00:00<00:00, 2923.28it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  55%|███████████████████▏               | 185/338 [00:00<00:00, 2935.68it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  55%|███████████████████▏               | 185/338 [00:00<00:00, 2933.20it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  55%|████████████          | 186/338 [00:00<00:00, 2945.57it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  55%|████████████          | 186/338 [00:00<00:00, 2943.08it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  55%|█████████████████▋              | 187/338 [00:00<00:00, 2955.36it/s, Materializing param=model.layers.15.self_attn.k_proj.bias]
Loading weights:  55%|█████████████████▋              | 187/338 [00:00<00:00, 2952.86it/s, Materializing param=model.layers.15.self_attn.k_proj.bias]
Loading weights:  56%|████████████████▋             | 188/338 [00:00<00:00, 2965.02it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  56%|████████████████▋             | 188/338 [00:00<00:00, 2962.54it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  56%|████████████████▊             | 189/338 [00:00<00:00, 2974.57it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  56%|████████████████▊             | 189/338 [00:00<00:00, 2972.11it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  56%|█████████████████▉              | 190/338 [00:00<00:00, 2984.26it/s, Materializing param=model.layers.15.self_attn.q_proj.bias]
Loading weights:  56%|█████████████████▉              | 190/338 [00:00<00:00, 2981.27it/s, Materializing param=model.layers.15.self_attn.q_proj.bias]
Loading weights:  57%|████████████████▉             | 191/338 [00:00<00:00, 2993.26it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  57%|████████████████▉             | 191/338 [00:00<00:00, 2990.80it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  57%|██████████████████▏             | 192/338 [00:00<00:00, 3002.98it/s, Materializing param=model.layers.15.self_attn.v_proj.bias]
Loading weights:  57%|██████████████████▏             | 192/338 [00:00<00:00, 3000.52it/s, Materializing param=model.layers.15.self_attn.v_proj.bias]
Loading weights:  57%|█████████████████▏            | 193/338 [00:00<00:00, 3012.59it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  57%|█████████████████▏            | 193/338 [00:00<00:00, 3009.78it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  57%|█████████████████▊             | 194/338 [00:00<00:00, 3021.80it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  57%|█████████████████▊             | 194/338 [00:00<00:00, 3019.31it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  58%|███████████████████              | 195/338 [00:00<00:00, 3031.38it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  58%|███████████████████              | 195/338 [00:00<00:00, 3028.92it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  58%|███████████████████▏             | 196/338 [00:00<00:00, 3040.81it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  58%|███████████████████▏             | 196/338 [00:00<00:00, 3038.31it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  58%|████████████████████▍              | 197/338 [00:00<00:00, 3050.25it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  58%|████████████████████▍              | 197/338 [00:00<00:00, 3047.80it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  59%|████████████▉         | 198/338 [00:00<00:00, 3059.24it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  59%|████████████▉         | 198/338 [00:00<00:00, 3056.62it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  59%|██████████████████▊             | 199/338 [00:00<00:00, 3068.55it/s, Materializing param=model.layers.16.self_attn.k_proj.bias]
Loading weights:  59%|██████████████████▊             | 199/338 [00:00<00:00, 3066.10it/s, Materializing param=model.layers.16.self_attn.k_proj.bias]
Loading weights:  59%|█████████████████▊            | 200/338 [00:00<00:00, 3077.94it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  59%|█████████████████▊            | 200/338 [00:00<00:00, 3075.45it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  59%|█████████████████▊            | 201/338 [00:00<00:00, 3087.27it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  59%|█████████████████▊            | 201/338 [00:00<00:00, 3084.77it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  60%|███████████████████             | 202/338 [00:00<00:00, 3096.43it/s, Materializing param=model.layers.16.self_attn.q_proj.bias]
Loading weights:  60%|███████████████████             | 202/338 [00:00<00:00, 3093.98it/s, Materializing param=model.layers.16.self_attn.q_proj.bias]
Loading weights:  60%|██████████████████            | 203/338 [00:00<00:00, 3105.72it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  60%|██████████████████            | 203/338 [00:00<00:00, 3103.26it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  60%|███████████████████▎            | 204/338 [00:00<00:00, 3115.13it/s, Materializing param=model.layers.16.self_attn.v_proj.bias]
Loading weights:  60%|███████████████████▎            | 204/338 [00:00<00:00, 3112.53it/s, Materializing param=model.layers.16.self_attn.v_proj.bias]
Loading weights:  61%|██████████████████▏           | 205/338 [00:00<00:00, 3124.22it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  61%|██████████████████▏           | 205/338 [00:00<00:00, 3121.68it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  61%|██████████████████▉            | 206/338 [00:00<00:00, 3132.82it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights:  61%|██████████████████▉            | 206/338 [00:00<00:00, 3130.27it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights:  61%|████████████████████▏            | 207/338 [00:00<00:00, 3141.89it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  61%|████████████████████▏            | 207/338 [00:00<00:00, 3139.40it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  62%|████████████████████▎            | 208/338 [00:00<00:00, 3150.92it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  62%|████████████████████▎            | 208/338 [00:00<00:00, 3148.47it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  62%|█████████████████████▋             | 209/338 [00:00<00:00, 3160.05it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  62%|████████████���████████▋             | 209/338 [00:00<00:00, 3157.59it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  62%|█████████████▋        | 210/338 [00:00<00:00, 3169.19it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  62%|█████████████▋        | 210/338 [00:00<00:00, 3166.60it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  62%|███████████████████▉            | 211/338 [00:00<00:00, 3178.24it/s, Materializing param=model.layers.17.self_attn.k_proj.bias]
Loading weights:  62%|███████████████████▉            | 211/338 [00:00<00:00, 3175.84it/s, Materializing param=model.layers.17.self_attn.k_proj.bias]
Loading weights:  63%|██████████████████▊           | 212/338 [00:00<00:00, 3187.31it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  63%|██████████████████▊           | 212/338 [00:00<00:00, 3184.79it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  63%|██████████████████▉           | 213/338 [00:00<00:00, 3196.19it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  63%|██████████████████▉           | 213/338 [00:00<00:00, 3193.08it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  63%|████████████████████▎           | 214/338 [00:00<00:00, 3204.28it/s, Materializing param=model.layers.17.self_attn.q_proj.bias]
Loading weights:  63%|████████████████████▎           | 214/338 [00:00<00:00, 3201.71it/s, Materializing param=model.layers.17.self_attn.q_proj.bias]
Loading weights:  64%|███████████████████           | 215/338 [00:00<00:00, 3213.08it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  64%|███████████████████           | 215/338 [00:00<00:00, 3210.41it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  64%|████████████████████▍           | 216/338 [00:00<00:00, 3221.76it/s, Materializing param=model.layers.17.self_attn.v_proj.bias]
Loading weights:  64%|████████████████████▍           | 216/338 [00:00<00:00, 3219.28it/s, Materializing param=model.layers.17.self_attn.v_proj.bias]
Loading weights:  64%|███████████████████▎          | 217/338 [00:00<00:00, 3230.65it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  64%|███████████████████▎          | 217/338 [00:00<00:00, 3228.18it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  64%|███████████████████▉           | 218/338 [00:00<00:00, 3239.46it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights:  64%|███████████████████▉           | 218/338 [00:00<00:00, 3236.95it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights:  65%|█████████████████████▍           | 219/338 [00:00<00:00, 3248.18it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  65%|█████████████████████▍           | 219/338 [00:00<00:00, 3245.67it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  65%|█████████████████████▍           | 220/338 [00:00<00:00, 3256.87it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  65%|█████████████████████▍           | 220/338 [00:00<00:00, 3254.32it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  65%|██████████████████████▉            | 221/338 [00:00<00:00, 3265.21it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  65%|██████████████████████▉            | 221/338 [00:00<00:00, 3262.44it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  66%|██████████████▍       | 222/338 [00:00<00:00, 3273.30it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  66%|██████████████▍       | 222/338 [00:00<00:00, 3270.62it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  66%|██████████████���██████           | 223/338 [00:00<00:00, 3281.33it/s, Materializing param=model.layers.18.self_attn.k_proj.bias]
Loading weights:  66%|█████████████████████           | 223/338 [00:00<00:00, 3278.58it/s, Materializing param=model.layers.18.self_attn.k_proj.bias]
Loading weights:  66%|███████████████████▉          | 224/338 [00:00<00:00, 3289.54it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  66%|███████████████████▉          | 224/338 [00:00<00:00, 3286.95it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  67%|███████████████████▉          | 225/338 [00:00<00:00, 3297.97it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  67%|███████████████████▉          | 225/338 [00:00<00:00, 3295.37it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  67%|█████████████████████▍          | 226/338 [00:00<00:00, 3306.09it/s, Materializing param=model.layers.18.self_attn.q_proj.bias]
Loading weights:  67%|█████████████████████▍          | 226/338 [00:00<00:00, 3303.18it/s, Materializing param=model.layers.18.self_attn.q_proj.bias]
Loading weights:  67%|████████████████████▏         | 227/338 [00:00<00:00, 3314.17it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  67%|████████████████████▏         | 227/338 [00:00<00:00, 3311.56it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  67%|█████████████████████▌          | 228/338 [00:00<00:00, 3322.51it/s, Materializing param=model.layers.18.self_attn.v_proj.bias]
Loading weights:  67%|█████████████████████▌          | 228/338 [00:00<00:00, 3319.61it/s, Materializing param=model.layers.18.self_attn.v_proj.bias]
Loading weights:  68%|████████████████████▎         | 229/338 [00:00<00:00, 3330.23it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  68%|████████████████████▎         | 229/338 [00:00<00:00, 3327.67it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  68%|█████████████████████          | 230/338 [00:00<00:00, 3338.51it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights:  68%|█████████████████████          | 230/338 [00:00<00:00, 3335.85it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights:  68%|██████████████████████▌          | 231/338 [00:00<00:00, 3346.67it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  68%|██████████████████████▌          | 231/338 [00:00<00:00, 3343.87it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  69%|██████████████████████▋          | 232/338 [00:00<00:00, 3354.67it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  69%|██████████████████████▋          | 232/338 [00:00<00:00, 3352.13it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  69%|████████████████████████▏          | 233/338 [00:00<00:00, 3363.01it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  69%|████████████████████████▏          | 233/338 [00:00<00:00, 3360.46it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  69%|███████████████▏      | 234/338 [00:00<00:00, 3371.35it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  69%|███████████████▏      | 234/338 [00:00<00:00, 3368.80it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  70%|██████████████████████▏         | 235/338 [00:00<00:00, 3379.65it/s, Materializing param=model.layers.19.self_attn.k_proj.bias]
Loading weights:  70%|██████████████████████▏         | 235/338 [00:00<00:00, 3377.04it/s, Materializing param=model.layers.19.self_attn.k_proj.bias]
Loading weights:  70%|████████████████████▉         | 236/338 [00:00<00:00, 3387.76it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  70%|████████████████████▉         | 236/338 [00:00<00:00, 3384.60it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  70%|█████████████████████         | 237/338 [00:00<00:00, 3394.84it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  70%|█████████████████████         | 237/338 [00:00<00:00, 3392.13it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  70%|██████████████████████▌         | 238/338 [00:00<00:00, 3402.77it/s, Materializing param=model.layers.19.self_attn.q_proj.bias]
Loading weights:  70%|██████████████████████▌         | 238/338 [00:00<00:00, 3400.20it/s, Materializing param=model.layers.19.self_attn.q_proj.bias]
Loading weights:  71%|█████████████████████▏        | 239/338 [00:00<00:00, 3410.83it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  71%|█████████████████████▏        | 239/338 [00:00<00:00, 3408.17it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  71%|██████████████████████▋         | 240/338 [00:00<00:00, 3418.82it/s, Materializing param=model.layers.19.self_attn.v_proj.bias]
Loading weights:  71%|██████████████████████▋         | 240/338 [00:00<00:00, 3416.21it/s, Materializing param=model.layers.19.self_attn.v_proj.bias]
Loading weights:  71%|█████████████████████▍        | 241/338 [00:00<00:00, 3426.80it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  71%|█████████████████████▍        | 241/338 [00:00<00:00, 3424.03it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  72%|██████████████████████▏        | 242/338 [00:00<00:00, 3434.55it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights:  72%|██████████████████████▏        | 242/338 [00:00<00:00, 3431.87it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights:  72%|███████████████████████▋         | 243/338 [00:00<00:00, 3442.33it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  72%|███████████████████████▋         | 243/338 [00:00<00:00, 3439.78it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  72%|███████████████████████▊         | 244/338 [00:00<00:00, 3449.55it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  72%|███████████████████████▊         | 244/338 [00:00<00:00, 3446.81it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  72%|█████████████████████████▎         | 245/338 [00:00<00:00, 3457.24it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  72%|█████████████████████████▎         | 245/338 [00:00<00:00, 3454.62it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  73%|████████████████      | 246/338 [00:00<00:00, 3465.03it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  73%|████████████████      | 246/338 [00:00<00:00, 3462.30it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  73%|███████████████████████▍        | 247/338 [00:00<00:00, 3472.68it/s, Materializing param=model.layers.20.self_attn.k_proj.bias]
Loading weights:  73%|███████████████████████▍        | 247/338 [00:00<00:00, 3469.90it/s, Materializing param=model.layers.20.self_attn.k_proj.bias]
Loading weights:  73%|██████████████████████        | 248/338 [00:00<00:00, 3480.06it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  73%|██████████████████████        | 248/338 [00:00<00:00, 3477.48it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  74%|██████████████████████        | 249/338 [00:00<00:00, 3487.83it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  74%|████���█████████████████        | 249/338 [00:00<00:00, 3485.28it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  74%|███████████████████████▋        | 250/338 [00:00<00:00, 3495.64it/s, Materializing param=model.layers.20.self_attn.q_proj.bias]
Loading weights:  74%|███████████████████████▋        | 250/338 [00:00<00:00, 3493.11it/s, Materializing param=model.layers.20.self_attn.q_proj.bias]
Loading weights:  74%|██████████████████████▎       | 251/338 [00:00<00:00, 3503.47it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  74%|██████████████████████▎       | 251/338 [00:00<00:00, 3500.37it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  75%|███████████████████████▊        | 252/338 [00:00<00:00, 3510.57it/s, Materializing param=model.layers.20.self_attn.v_proj.bias]
Loading weights:  75%|███████████████████████▊        | 252/338 [00:00<00:00, 3507.99it/s, Materializing param=model.layers.20.self_attn.v_proj.bias]
Loading weights:  75%|██████████████████████▍       | 253/338 [00:00<00:00, 3518.28it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  75%|██████████████████████▍       | 253/338 [00:00<00:00, 3515.70it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  75%|███████████████████████▎       | 254/338 [00:00<00:00, 3525.94it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights:  75%|███████████████████████▎       | 254/338 [00:00<00:00, 3523.19it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights:  75%|████████████████████████▉        | 255/338 [00:00<00:00, 3533.28it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  75%|████████████████████████▉        | 255/338 [00:00<00:00, 3530.65it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  76%|████████████████████████▉        | 256/338 [00:00<00:00, 3540.79it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  76%|████████████████████████▉        | 256/338 [00:00<00:00, 3538.23it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  76%|██████████████████████████▌        | 257/338 [00:00<00:00, 3548.42it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  76%|██████████████████████████▌        | 257/338 [00:00<00:00, 3545.70it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  76%|████████████████▊     | 258/338 [00:00<00:00, 3555.67it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  76%|████████████████▊     | 258/338 [00:00<00:00, 3553.06it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  77%|████████████████████████▌       | 259/338 [00:00<00:00, 3562.75it/s, Materializing param=model.layers.21.self_attn.k_proj.bias]
Loading weights:  77%|████████████████████████▌       | 259/338 [00:00<00:00, 3559.85it/s, Materializing param=model.layers.21.self_attn.k_proj.bias]
Loading weights:  77%|███████████████████████       | 260/338 [00:00<00:00, 3569.77it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  77%|███████████████████████       | 260/338 [00:00<00:00, 3567.20it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  77%|███████████████████████▏      | 261/338 [00:00<00:00, 3577.33it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  77%|███████████████████████▏      | 261/338 [00:00<00:00, 3574.80it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  78%|████████████████████████▊       | 262/338 [00:00<00:00, 3584.92it/s, Materializing param=model.layers.21.self_attn.q_proj.bias]
Loading weights:  78%|████████████████████████▊       | 262/338 [00:00<00:00, 3582.35it/s, Materializing param=model.layers.21.self_attn.q_proj.bias]
Loading weights:  78%|███████████████████████▎      | 263/338 [00:00<00:00, 3592.44it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  78%|███████████████████████▎      | 263/338 [00:00<00:00, 3589.94it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  78%|████████████████████████▉       | 264/338 [00:00<00:00, 3599.93it/s, Materializing param=model.layers.21.self_attn.v_proj.bias]
Loading weights:  78%|████████████████████████▉       | 264/338 [00:00<00:00, 3597.37it/s, Materializing param=model.layers.21.self_attn.v_proj.bias]
Loading weights:  78%|███████████████████████▌      | 265/338 [00:00<00:00, 3607.35it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  78%|███████████████████████▌      | 265/338 [00:00<00:00, 3604.73it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  79%|████████████████████████▍      | 266/338 [00:00<00:00, 3614.80it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights:  79%|████████████████████████▍      | 266/338 [00:00<00:00, 3612.28it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights:  79%|██████████████████████████       | 267/338 [00:00<00:00, 3621.79it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  79%|██████████████████████████       | 267/338 [00:00<00:00, 3619.09it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  79%|██████████████████████████▏      | 268/338 [00:00<00:00, 3628.70it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  79%|██████████████████████████▏      | 268/338 [00:00<00:00, 3626.09it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  80%|███████████████████████████▊       | 269/338 [00:00<00:00, 3635.98it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  80%|███████████████████████████▊       | 269/338 [00:00<00:00, 3633.40it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  80%|█████████████████▌    | 270/338 [00:00<00:00, 3643.25it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  80%|█████████████████▌    | 270/338 [00:00<00:00, 3640.61it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  80%|█████████████████████████▋      | 271/338 [00:00<00:00, 3650.47it/s, Materializing param=model.layers.22.self_attn.k_proj.bias]
Loading weights:  80%|█████████████████████████▋      | 271/338 [00:00<00:00, 3647.90it/s, Materializing param=model.layers.22.self_attn.k_proj.bias]
Loading weights:  80%|████████████████████████▏     | 272/338 [00:00<00:00, 3657.70it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  80%|████████████████████████▏     | 272/338 [00:00<00:00, 3655.11it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  81%|████████████████████████▏     | 273/338 [00:00<00:00, 3664.93it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  81%|████████████████████████▏     | 273/338 [00:00<00:00, 3662.32it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  81%|█████████████████████████▉      | 274/338 [00:00<00:00, 3672.11it/s, Materializing param=model.layers.22.self_attn.q_proj.bias]
Loading weights:  81%|█████████████████████████▉      | 274/338 [00:00<00:00, 3669.55it/s, Materializing param=model.layers.22.self_attn.q_proj.bias]
Loading weights:  81%|████████████████████████▍     | 275/338 [00:00<00:00, 3678.67it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  81%|████████████████████████▍     | 275/338 [00:00<00:00, 3676.04it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  82%|██████████████████████████▏     | 276/338 [00:00<00:00, 3685.88it/s, Materializing param=model.layers.22.self_attn.v_proj.bias]
Loading weights:  82%|██████████████████████████▏     | 276/338 [00:00<00:00, 3683.35it/s, Materializing param=model.layers.22.self_attn.v_proj.bias]
Loading weights:  82%|████████████████████████▌     | 277/338 [00:00<00:00, 3693.13it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  82%|████████████████████████▌     | 277/338 [00:00<00:00, 3690.59it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  82%|█████████████████████████▍     | 278/338 [00:00<00:00, 3700.16it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights:  82%|█████████████████████████▍     | 278/338 [00:00<00:00, 3697.59it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights:  83%|███████████████████████████▏     | 279/338 [00:00<00:00, 3707.30it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  83%|███████████████████████████▏     | 279/338 [00:00<00:00, 3704.69it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  83%|███████████████████████████▎     | 280/338 [00:00<00:00, 3714.36it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  83%|███████████████████████████▎     | 280/338 [00:00<00:00, 3711.80it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  83%|█████████████████████████████      | 281/338 [00:00<00:00, 3721.47it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  83%|█████████████████████████████      | 281/338 [00:00<00:00, 3718.93it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  83%|██████████████████▎   | 282/338 [00:00<00:00, 3728.65it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  83%|██████████████████▎   | 282/338 [00:00<00:00, 3725.65it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  84%|██████████████████████████▊     | 283/338 [00:00<00:00, 3735.19it/s, Materializing param=model.layers.23.self_attn.k_proj.bias]
Loading weights:  84%|██████████████████████████▊     | 283/338 [00:00<00:00, 3732.69it/s, Materializing param=model.layers.23.self_attn.k_proj.bias]
Loading weights:  84%|█████████████████████████▏    | 284/338 [00:00<00:00, 3742.30it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  84%|█████████████████████████▏    | 284/338 [00:00<00:00, 3739.65it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  84%|█████████████████████████▎    | 285/338 [00:00<00:00, 3749.24it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  84%|█████████████████████████▎    | 285/338 [00:00<00:00, 3746.64it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  85%|███████████████████████████     | 286/338 [00:00<00:00, 3756.18it/s, Materializing param=model.layers.23.self_attn.q_proj.bias]
Loading weights:  85%|███████████████████████████     | 286/338 [00:00<00:00, 3753.57it/s, Materializing param=model.layers.23.self_attn.q_proj.bias]
Loading weights:  85%|█████████████████████████▍    | 287/338 [00:00<00:00, 3763.08it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  85%|█████████████████████████▍    | 287/338 [00:00<00:00, 3760.44it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  85%|███████████████████████████▎    | 288/338 [00:00<00:00, 3770.04it/s, Materializing param=model.layers.23.self_attn.v_proj.bias]
Loading weights:  85%|███████████████████████████▎    | 288/338 [00:00<00:00, 3767.21it/s, Materializing param=model.layers.23.self_attn.v_proj.bias]
Loading weights:  86%|█████████████████████████▋    | 289/338 [00:00<00:00, 3776.65it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  86%|█████████████████████████▋    | 289/338 [00:00<00:00, 3774.05it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  86%|██████████████████████████▌    | 290/338 [00:00<00:00, 3783.57it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights:  86%|██████████████████████████▌    | 290/338 [00:00<00:00, 3780.47it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights:  86%|████████████████████████████▍    | 291/338 [00:00<00:00, 3789.77it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  86%|████████████████████████████▍    | 291/338 [00:00<00:00, 3787.21it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  86%|████████████████████████████▌    | 292/338 [00:00<00:00, 3796.64it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  86%|████████████████████████████▌    | 292/338 [00:00<00:00, 3794.06it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  87%|██████████████████████████████▎    | 293/338 [00:00<00:00, 3803.43it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  87%|██████████████████████████████▎    | 293/338 [00:00<00:00, 3800.88it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  87%|███████████████████▏  | 294/338 [00:00<00:00, 3810.27it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  87%|███████████████████▏  | 294/338 [00:00<00:00, 3807.65it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  87%|███████████████████████████▉    | 295/338 [00:00<00:00, 3816.99it/s, Materializing param=model.layers.24.self_attn.k_proj.bias]
Loading weights:  87%|███████████████████████████▉    | 295/338 [00:00<00:00, 3814.36it/s, Materializing param=model.layers.24.self_attn.k_proj.bias]
Loading weights:  88%|██████████████████████████▎   | 296/338 [00:00<00:00, 3823.62it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  88%|██████████████████████████▎   | 296/338 [00:00<00:00, 3821.02it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  88%|██████████████████████████▎   | 297/338 [00:00<00:00, 3830.32it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  88%|██████████████████████████▎   | 297/338 [00:00<00:00, 3827.68it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  88%|████████████████████████████▏   | 298/338 [00:00<00:00, 3836.76it/s, Materializing param=model.layers.24.self_attn.q_proj.bias]
Loading weights:  88%|████████████████████████████▏   | 298/338 [00:00<00:00, 3833.41it/s, Materializing param=model.layers.24.self_attn.q_proj.bias]
Loading weights:  88%|██████████████████████████▌   | 299/338 [00:00<00:00, 3842.39it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  88%|██████████████████████████▌   | 299/338 [00:00<00:00, 3839.69it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  89%|████████████████████████████▍   | 300/338 [00:00<00:00, 3848.97it/s, Materializing param=model.layers.24.self_attn.v_proj.bias]
Loading weights:  89%|████████████████████████████▍   | 300/338 [00:00<00:00, 3846.41it/s, Materializing param=model.layers.24.self_attn.v_proj.bias]
Loading weights:  89%|██████████████████████████▋   | 301/338 [00:00<00:00, 3855.60it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  89%|██████████████████████████▋   | 301/338 [00:00<00:00, 3848.99it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  89%|███████████████████████████▋   | 302/338 [00:00<00:00, 3857.47it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights:  89%|███████████████████████████▋   | 302/338 [00:00<00:00, 3854.84it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights:  90%|█████████████████████████████▌   | 303/338 [00:00<00:00, 3863.85it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  90%|█████████████████████████████▌   | 303/338 [00:00<00:00, 3861.30it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  90%|█████████████████████████████▋   | 304/338 [00:00<00:00, 3870.27it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  90%|█████████████████████████████▋   | 304/338 [00:00<00:00, 3867.75it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  90%|███████████████████████████████▌   | 305/338 [00:00<00:00, 3876.91it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  90%|███████████████████████████████▌   | 305/338 [00:00<00:00, 3873.78it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  91%|███████████████████▉  | 306/338 [00:00<00:00, 3882.66it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  91%|███████████████████▉  | 306/338 [00:00<00:00, 3879.93it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  91%|█████████████████████████████   | 307/338 [00:00<00:00, 3889.02it/s, Materializing param=model.layers.25.self_attn.k_proj.bias]
Loading weights:  91%|█████████████████████████████   | 307/338 [00:00<00:00, 3886.51it/s, Materializing param=model.layers.25.self_attn.k_proj.bias]
Loading weights:  91%|███████████████████████████▎  | 308/338 [00:00<00:00, 3895.57it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  91%|███████████████████████████▎  | 308/338 [00:00<00:00, 3892.81it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  91%|███████████████████████████▍  | 309/338 [00:00<00:00, 3901.69it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  91%|███████████████████████████▍  | 309/338 [00:00<00:00, 3899.10it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  92%|█████████████████████████████▎  | 310/338 [00:00<00:00, 3908.06it/s, Materializing param=model.layers.25.self_attn.q_proj.bias]
Loading weights:  92%|█████████████████████████████▎  | 310/338 [00:00<00:00, 3905.49it/s, Materializing param=model.layers.25.self_attn.q_proj.bias]
Loading weights:  92%|███████████████████████████▌  | 311/338 [00:00<00:00, 3914.46it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  92%|███████████████████████████▌  | 311/338 [00:00<00:00, 3911.94it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  92%|█████████████████████████████▌  | 312/338 [00:00<00:00, 3920.92it/s, Materializing param=model.layers.25.self_attn.v_proj.bias]
Loading weights:  92%|█████████████████████████████▌  | 312/338 [00:00<00:00, 3918.35it/s, Materializing param=model.layers.25.self_attn.v_proj.bias]
Loading weights:  93%|███████████████████████████▊  | 313/338 [00:00<00:00, 3926.83it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  93%|███████████████████████████▊  | 313/338 [00:00<00:00, 3923.99it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  93%|████████████████████████████▊  | 314/338 [00:00<00:00, 3932.81it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights:  93%|████████████████████████████▊  | 314/338 [00:00<00:00, 3930.25it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights:  93%|██████████████████████████████▊  | 315/338 [00:00<00:00, 3939.10it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  93%|██████████████████████████████▊  | 315/338 [00:00<00:00, 3936.49it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  93%|██████████████████████████████▊  | 316/338 [00:00<00:00, 3945.30it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  93%|██████████████████████████████▊  | 316/338 [00:00<00:00, 3942.72it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  94%|████████████████████████████████▊  | 317/338 [00:00<00:00, 3951.54it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  94%|████████████████████████████████▊  | 317/338 [00:00<00:00, 3949.02it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  94%|████████████████████▋ | 318/338 [00:00<00:00, 3957.64it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  94%|████████████████████▋ | 318/338 [00:00<00:00, 3954.91it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  94%|██████████████████████████████▏ | 319/338 [00:00<00:00, 3963.63it/s, Materializing param=model.layers.26.self_attn.k_proj.bias]
Loading weights:  94%|██████████████████████████████▏ | 319/338 [00:00<00:00, 3961.11it/s, Materializing param=model.layers.26.self_attn.k_proj.bias]
Loading weights:  95%|████████████████████████████▍ | 320/338 [00:00<00:00, 3969.74it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  95%|████████████████████████████▍ | 320/338 [00:00<00:00, 3967.15it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  95%|████████████████████████████▍ | 321/338 [00:00<00:00, 3975.26it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  95%|████████████████████████████▍ | 321/338 [00:00<00:00, 3972.55it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  95%|██████████████████████████████▍ | 322/338 [00:00<00:00, 3981.09it/s, Materializing param=model.layers.26.self_attn.q_proj.bias]
Loading weights:  95%|██████████████████████████████▍ | 322/338 [00:00<00:00, 3978.49it/s, Materializing param=model.layers.26.self_attn.q_proj.bias]
Loading weights:  96%|████████████████████████████▋ | 323/338 [00:00<00:00, 3987.07it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  96%|████████████████████████████▋ | 323/338 [00:00<00:00, 3984.53it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  96%|██████████████████████████████▋ | 324/338 [00:00<00:00, 3993.28it/s, Materializing param=model.layers.26.self_attn.v_proj.bias]
Loading weights:  96%|██████████████████████████████▋ | 324/338 [00:00<00:00, 3990.72it/s, Materializing param=model.layers.26.self_attn.v_proj.bias]
Loading weights:  96%|████████████████████████████▊ | 325/338 [00:00<00:00, 3999.39it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  96%|████████████████████████████▊ | 325/338 [00:00<00:00, 3996.84it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  96%|█████████████████████████████▉ | 326/338 [00:00<00:00, 4005.48it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights:  96%|█████████████████████████████▉ | 326/338 [00:00<00:00, 4002.92it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights:  97%|███████████████████████████████▉ | 327/338 [00:00<00:00, 4011.43it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  97%|███████████████████████████████▉ | 327/338 [00:00<00:00, 4008.85it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  97%|████████████████████████████████ | 328/338 [00:00<00:00, 4017.27it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  97%|████████████████████████████████ | 328/338 [00:00<00:00, 4014.09it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  97%|██████████████████████████████████ | 329/338 [00:00<00:00, 4022.56it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  97%|██████████████████████████████████ | 329/338 [00:00<00:00, 4019.97it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  98%|█████████████████████▍| 330/338 [00:00<00:00, 4028.55it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  98%|█████████████████████▍| 330/338 [00:00<00:00, 4025.96it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  98%|███████████████████████████████▎| 331/338 [00:00<00:00, 4034.60it/s, Materializing param=model.layers.27.self_attn.k_proj.bias]
Loading weights:  98%|███████████████████████████████▎| 331/338 [00:00<00:00, 4032.02it/s, Materializing param=model.layers.27.self_attn.k_proj.bias]
Loading weights:  98%|█████████████████████████████▍| 332/338 [00:00<00:00, 4040.65it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  98%|█████████████████████████████▍| 332/338 [00:00<00:00, 4038.13it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  99%|█████████████████████████████▌| 333/338 [00:00<00:00, 4046.76it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  99%|█████████████████████████████▌| 333/338 [00:00<00:00, 4044.23it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  99%|███████████████████████████████▌| 334/338 [00:00<00:00, 4052.74it/s, Materializing param=model.layers.27.self_attn.q_proj.bias]
Loading weights:  99%|███████████████████████████████▌| 334/338 [00:00<00:00, 4050.21it/s, Materializing param=model.layers.27.self_attn.q_proj.bias]
Loading weights:  99%|█████████████████████████████▋| 335/338 [00:00<00:00, 4058.73it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  99%|█████████████████████████████▋| 335/338 [00:00<00:00, 4056.22it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  99%|███████████████████████████████▊| 336/338 [00:00<00:00, 4064.83it/s, Materializing param=model.layers.27.self_attn.v_proj.bias]
Loading weights:  99%|███████████████████████████████▊| 336/338 [00:00<00:00, 4061.82it/s, Materializing param=model.layers.27.self_attn.v_proj.bias]
Loading weights: 100%|█████████████████████████████▉| 337/338 [00:00<00:00, 4070.25it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 100%|█████████████████████████████▉| 337/338 [00:00<00:00, 4067.56it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 100%|████████████████████████████████████████████████████| 338/338 [00:00<00:00, 4075.93it/s, Materializing param=model.norm.weight]
Loading weights: 100%|████████████████████████████████████████████████████| 338/338 [00:00<00:00, 4073.42it/s, Materializing param=model.norm.weight]
Loading weights: 100%|████████████████████████████████████████████████████| 338/338 [00:00<00:00, 4068.04it/s, Materializing param=model.norm.weight]
-[2026-02-10 06:24:29,893] [WARNING] [torchao.<module>:39] [PID:52829] Skipping import of cpp extensions due to incompatible torch version 2.9.1+cu128 for torchao version 0.13.0
-[2026-02-10 06:24:34,917] [WARNING] [accelerate.utils.dataclasses.__post_init__:1962] [PID:52829] sharding_strategy is deprecated in favor of reshard_after_forward. This will be removed in a future version of Accelerate.
-[2026-02-10 07:02:45,775] [WARNING] [py.warnings._showwarnmsg:110] [PID:52829] /root/miniconda3/envs/py3.11/lib/python3.11/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html .
-  warnings.warn(
-
-[2026-02-10 07:40:55,009] [WARNING] [py.warnings._showwarnmsg:110] [PID:52829] /root/miniconda3/envs/py3.11/lib/python3.11/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html .
+
Loading dataset from disk:   0%|                                                              | 0/205 [00:00<?, ?it/s]
Loading dataset from disk: 100%|████████████████████████████████████████████████| 205/205 [00:00<00:00, 279620.27it/s]
+
Loading weights:   0%|                                                                        | 0/338 [00:00<?, ?it/s]
Loading weights:   0%|              | 1/338 [00:00<00:00, 13443.28it/s, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|               | 1/338 [00:00<00:00, 6584.46it/s, Materializing param=model.embed_tokens.weight]
Loading weights:   1%|   | 2/338 [00:00<00:00, 6553.60it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|   | 2/338 [00:00<00:00, 5366.99it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|     | 3/338 [00:00<00:00, 5971.96it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   1%|     | 3/338 [00:00<00:00, 5123.34it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   1%|     | 4/338 [00:00<00:00, 5727.97it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|     | 4/338 [00:00<00:00, 5287.49it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|       | 5/338 [00:00<00:00, 4604.07it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   1%|       | 5/338 [00:00<00:00, 4369.98it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   2%| | 6/338 [00:00<00:00, 4835.86it/s, Materializing param=model.layers.0.post_attention_layernorm.
Loading weights:   2%| | 6/338 [00:00<00:00, 4637.15it/s, Materializing param=model.layers.0.post_attention_layernorm.
Loading weights:   2%|    | 7/338 [00:00<00:00, 5006.84it/s, Materializing param=model.layers.0.self_attn.k_proj.bias]
Loading weights:   2%|    | 7/338 [00:00<00:00, 4822.62it/s, Materializing param=model.layers.0.self_attn.k_proj.bias]
Loading weights:   2%|  | 8/338 [00:00<00:00, 5189.36it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|  | 8/338 [00:00<00:00, 5021.62it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   3%|  | 9/338 [00:00<00:00, 5292.87it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   3%|  | 9/338 [00:00<00:00, 5131.69it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   3%|   | 10/338 [00:00<00:00, 5437.26it/s, Materializing param=model.layers.0.self_attn.q_proj.bias]
Loading weights:   3%|   | 10/338 [00:00<00:00, 5287.16it/s, Materializing param=model.layers.0.self_attn.q_proj.bias]
Loading weights:   3%| | 11/338 [00:00<00:00, 5520.80it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%| | 11/338 [00:00<00:00, 5382.96it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   4%|   | 12/338 [00:00<00:00, 5634.35it/s, Materializing param=model.layers.0.self_attn.v_proj.bias]
Loading weights:   4%|   | 12/338 [00:00<00:00, 5495.92it/s, Materializing param=model.layers.0.self_attn.v_proj.bias]
Loading weights:   4%| | 13/338 [00:00<00:00, 5690.46it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   4%| | 13/338 [00:00<00:00, 5565.58it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   4%|  | 14/338 [00:00<00:00, 5588.68it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   4%|  | 14/338 [00:00<00:00, 5469.47it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   4%|▏   | 15/338 [00:00<00:00, 5672.07it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   4%|▏   | 15/338 [00:00<00:00, 5557.82it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   5%|▏   | 16/338 [00:00<00:00, 5675.65it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   5%|▏   | 16/338 [00:00<00:00, 5303.37it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   5%|▎     | 17/338 [00:00<00:00, 5436.35it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   5%|▎     | 17/338 [00:00<00:00, 5339.06it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   5%| | 18/338 [00:00<00:00, 5498.72it/s, Materializing param=model.layers.1.post_attention_layernorm
Loading weights:   5%| | 18/338 [00:00<00:00, 5410.45it/s, Materializing param=model.layers.1.post_attention_layernorm
Loading weights:   6%|▏  | 19/338 [00:00<00:00, 5413.84it/s, Materializing param=model.layers.1.self_attn.k_proj.bias]
Loading weights:   6%|▏  | 19/338 [00:00<00:00, 5331.27it/s, Materializing param=model.layers.1.self_attn.k_proj.bias]
Loading weights:   6%| | 20/338 [00:00<00:00, 5478.10it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   6%| | 20/338 [00:00<00:00, 5403.64it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   6%| | 21/338 [00:00<00:00, 5301.58it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   6%| | 21/338 [00:00<00:00, 5226.39it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   7%|▏  | 22/338 [00:00<00:00, 4713.18it/s, Materializing param=model.layers.1.self_attn.q_proj.bias]
Loading weights:   7%|▏  | 22/338 [00:00<00:00, 4654.23it/s, Materializing param=model.layers.1.self_attn.q_proj.bias]
Loading weights:   7%| | 23/338 [00:00<00:00, 4631.03it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   7%| | 23/338 [00:00<00:00, 4582.41it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   7%|▏  | 24/338 [00:00<00:00, 4701.91it/s, Materializing param=model.layers.1.self_attn.v_proj.bias]
Loading weights:   7%|▏  | 24/338 [00:00<00:00, 4655.60it/s, Materializing param=model.layers.1.self_attn.v_proj.bias]
Loading weights:   7%| | 25/338 [00:00<00:00, 4769.29it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   7%| | 25/338 [00:00<00:00, 4722.46it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   8%|▏ | 26/338 [00:00<00:00, 4710.06it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   8%|▏ | 26/338 [00:00<00:00, 4663.53it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   8%|▎   | 27/338 [00:00<00:00, 4674.96it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   8%|▎   | 27/338 [00:00<00:00, 4629.85it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   8%|▎   | 28/338 [00:00<00:00, 4651.11it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   8%|▎   | 28/338 [00:00<00:00, 4610.21it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   9%|▌     | 29/338 [00:00<00:00, 4705.95it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   9%|▌     | 29/338 [00:00<00:00, 4665.16it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   9%| | 30/338 [00:00<00:00, 4759.76it/s, Materializing param=model.layers.2.post_attention_layernorm
Loading weights:   9%| | 30/338 [00:00<00:00, 4719.06it/s, Materializing param=model.layers.2.post_attention_layernorm
Loading weights:   9%|▎  | 31/338 [00:00<00:00, 4777.46it/s, Materializing param=model.layers.2.self_attn.k_proj.bias]
Loading weights:   9%|▎  | 31/338 [00:00<00:00, 4738.98it/s, Materializing param=model.layers.2.self_attn.k_proj.bias]
Loading weights:   9%| | 32/338 [00:00<00:00, 4673.81it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   9%| | 32/338 [00:00<00:00, 4637.15it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:  10%| | 33/338 [00:00<00:00, 4721.87it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:  10%| | 33/338 [00:00<00:00, 4639.87it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:  10%|▎  | 34/338 [00:00<00:00, 4485.32it/s, Materializing param=model.layers.2.self_attn.q_proj.bias]
Loading weights:  10%|▎  | 34/338 [00:00<00:00, 4054.54it/s, Materializing param=model.layers.2.self_attn.q_proj.bias]
Loading weights:  10%| | 35/338 [00:00<00:00, 4059.08it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:  10%| | 35/338 [00:00<00:00, 4033.21it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:  11%|▎  | 36/338 [00:00<00:00, 4107.03it/s, Materializing param=model.layers.2.self_attn.v_proj.bias]
Loading weights:  11%|▎  | 36/338 [00:00<00:00, 4081.39it/s, Materializing param=model.layers.2.self_attn.v_proj.bias]
Loading weights:  11%| | 37/338 [00:00<00:00, 4089.52it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:  11%| | 37/338 [00:00<00:00, 4064.99it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:  11%|▏ | 38/338 [00:00<00:00, 4135.21it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:  11%|▏ | 38/338 [00:00<00:00, 4111.96it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:  12%|▍   | 39/338 [00:00<00:00, 4141.21it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:  12%|▍   | 39/338 [00:00<00:00, 4117.96it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:  12%|▍   | 40/338 [00:00<00:00, 4185.62it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:  12%|▍   | 40/338 [00:00<00:00, 4161.84it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:  12%|▋     | 41/338 [00:00<00:00, 4223.04it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:  12%|▋     | 41/338 [00:00<00:00, 4199.84it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:  12%| | 42/338 [00:00<00:00, 4259.62it/s, Materializing param=model.layers.3.post_attention_layernorm
Loading weights:  12%| | 42/338 [00:00<00:00, 4236.98it/s, Materializing param=model.layers.3.post_attention_layernorm
Loading weights:  13%|▍  | 43/338 [00:00<00:00, 4301.65it/s, Materializing param=model.layers.3.self_attn.k_proj.bias]
Loading weights:  13%|▍  | 43/338 [00:00<00:00, 4280.11it/s, Materializing param=model.layers.3.self_attn.k_proj.bias]
Loading weights:  13%|▏| 44/338 [00:00<00:00, 4186.88it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  13%|▏| 44/338 [00:00<00:00, 4163.46it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  13%|▏| 45/338 [00:00<00:00, 4222.45it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  13%|▏| 45/338 [00:00<00:00, 4200.84it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  14%|▍  | 46/338 [00:00<00:00, 4253.30it/s, Materializing param=model.layers.3.self_attn.q_proj.bias]
Loading weights:  14%|▍  | 46/338 [00:00<00:00, 4232.77it/s, Materializing param=model.layers.3.self_attn.q_proj.bias]
Loading weights:  14%|▏| 47/338 [00:00<00:00, 4291.55it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  14%|▏| 47/338 [00:00<00:00, 4272.39it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  14%|▍  | 48/338 [00:00<00:00, 4145.25it/s, Materializing param=model.layers.3.self_attn.v_proj.bias]
Loading weights:  14%|▍  | 48/338 [00:00<00:00, 4074.78it/s, Materializing param=model.layers.3.self_attn.v_proj.bias]
Loading weights:  14%|▏| 49/338 [00:00<00:00, 4123.03it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  14%|▏| 49/338 [00:00<00:00, 4104.84it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  15%|▎ | 50/338 [00:00<00:00, 4158.21it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  15%|▎ | 50/338 [00:00<00:00, 3999.83it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  15%|▌   | 51/338 [00:00<00:00, 4027.06it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  15%|▌   | 51/338 [00:00<00:00, 4010.00it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  15%|▌   | 52/338 [00:00<00:00, 4061.30it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  15%|▌   | 52/338 [00:00<00:00, 4045.03it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  16%|▉     | 53/338 [00:00<00:00, 3973.30it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  16%|▉     | 53/338 [00:00<00:00, 3956.75it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  16%|▏| 54/338 [00:00<00:00, 3984.67it/s, Materializing param=model.layers.4.post_attention_layernorm
Loading weights:  16%|▏| 54/338 [00:00<00:00, 3968.26it/s, Materializing param=model.layers.4.post_attention_layernorm
Loading weights:  16%|▍  | 55/338 [00:00<00:00, 4014.74it/s, Materializing param=model.layers.4.self_attn.k_proj.bias]
Loading weights:  16%|▍  | 55/338 [00:00<00:00, 3999.35it/s, Materializing param=model.layers.4.self_attn.k_proj.bias]
Loading weights:  17%|▏| 56/338 [00:00<00:00, 4047.02it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  17%|▏| 56/338 [00:00<00:00, 4031.95it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  17%|▏| 57/338 [00:00<00:00, 4074.15it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  17%|▏| 57/338 [00:00<00:00, 4059.07it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  17%|▌  | 58/338 [00:00<00:00, 4105.47it/s, Materializing param=model.layers.4.self_attn.q_proj.bias]
Loading weights:  17%|▌  | 58/338 [00:00<00:00, 4090.70it/s, Materializing param=model.layers.4.self_attn.q_proj.bias]
Loading weights:  17%|▏| 59/338 [00:00<00:00, 4120.21it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  17%|▏| 59/338 [00:00<00:00, 4102.38it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  18%|▌  | 60/338 [00:00<00:00, 4137.75it/s, Materializing param=model.layers.4.self_attn.v_proj.bias]
Loading weights:  18%|▌  | 60/338 [00:00<00:00, 4122.91it/s, Materializing param=model.layers.4.self_attn.v_proj.bias]
Loading weights:  18%|▏| 61/338 [00:00<00:00, 4166.64it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  18%|▏| 61/338 [00:00<00:00, 4151.90it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  18%|▎ | 62/338 [00:00<00:00, 4191.80it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  18%|▎ | 62/338 [00:00<00:00, 4176.59it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  19%|▋   | 63/338 [00:00<00:00, 4127.28it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  19%|▋   | 63/338 [00:00<00:00, 4112.06it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  19%|▊   | 64/338 [00:00<00:00, 4127.05it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  19%|▊   | 64/338 [00:00<00:00, 4112.13it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  19%|█▏    | 65/338 [00:00<00:00, 4149.43it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  19%|█▏    | 65/338 [00:00<00:00, 4135.01it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  20%|▏| 66/338 [00:00<00:00, 4175.51it/s, Materializing param=model.layers.5.post_attention_layernorm
Loading weights:  20%|▏| 66/338 [00:00<00:00, 4160.52it/s, Materializing param=model.layers.5.post_attention_layernorm
Loading weights:  20%|▌  | 67/338 [00:00<00:00, 4196.81it/s, Materializing param=model.layers.5.self_attn.k_proj.bias]
Loading weights:  20%|▌  | 67/338 [00:00<00:00, 4181.88it/s, Materializing param=model.layers.5.self_attn.k_proj.bias]
Loading weights:  20%|▏| 68/338 [00:00<00:00, 4178.63it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  20%|▏| 68/338 [00:00<00:00, 4163.99it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  20%|▏| 69/338 [00:00<00:00, 4147.12it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  20%|▏| 69/338 [00:00<00:00, 4132.91it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  21%|▌  | 70/338 [00:00<00:00, 4166.51it/s, Materializing param=model.layers.5.self_attn.q_proj.bias]
Loading weights:  21%|▌  | 70/338 [00:00<00:00, 4152.78it/s, Materializing param=model.layers.5.self_attn.q_proj.bias]
Loading weights:  21%|▏| 71/338 [00:00<00:00, 4110.53it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  21%|▏| 71/338 [00:00<00:00, 4097.13it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  21%|▋  | 72/338 [00:00<00:00, 4065.18it/s, Materializing param=model.layers.5.self_attn.v_proj.bias]
Loading weights:  21%|▋  | 72/338 [00:00<00:00, 4052.63it/s, Materializing param=model.layers.5.self_attn.v_proj.bias]
Loading weights:  22%|▏| 73/338 [00:00<00:00, 4089.27it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  22%|▏| 73/338 [00:00<00:00, 4077.18it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  22%|▍ | 74/338 [00:00<00:00, 4109.72it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  22%|▍ | 74/338 [00:00<00:00, 4098.33it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  22%|▉   | 75/338 [00:00<00:00, 4134.38it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  22%|▉   | 75/338 [00:00<00:00, 4122.41it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  22%|▉   | 76/338 [00:00<00:00, 4154.89it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  22%|▉   | 76/338 [00:00<00:00, 4143.33it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  23%|█▎    | 77/338 [00:00<00:00, 4170.80it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  23%|█▎    | 77/338 [00:00<00:00, 4156.95it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  23%|▏| 78/338 [00:00<00:00, 4169.39it/s, Materializing param=model.layers.6.post_attention_layernorm
Loading weights:  23%|▏| 78/338 [00:00<00:00, 4096.51it/s, Materializing param=model.layers.6.post_attention_layernorm
Loading weights:  23%|▋  | 79/338 [00:00<00:00, 4129.33it/s, Materializing param=model.layers.6.self_attn.k_proj.bias]
Loading weights:  23%|▋  | 79/338 [00:00<00:00, 4118.45it/s, Materializing param=model.layers.6.self_attn.k_proj.bias]
Loading weights:  24%|▏| 80/338 [00:00<00:00, 4045.33it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  24%|▏| 80/338 [00:00<00:00, 4033.42it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  24%|▏| 81/338 [00:00<00:00, 3980.30it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  24%|▏| 81/338 [00:00<00:00, 3969.51it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  24%|▋  | 82/338 [00:00<00:00, 3998.99it/s, Materializing param=model.layers.6.self_attn.q_proj.bias]
Loading weights:  24%|▋  | 82/338 [00:00<00:00, 3988.51it/s, Materializing param=model.layers.6.self_attn.q_proj.bias]
Loading weights:  25%|▏| 83/338 [00:00<00:00, 4019.71it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  25%|▏| 83/338 [00:00<00:00, 4009.44it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  25%|▋  | 84/338 [00:00<00:00, 4038.35it/s, Materializing param=model.layers.6.self_attn.v_proj.bias]
Loading weights:  25%|▋  | 84/338 [00:00<00:00, 4027.68it/s, Materializing param=model.layers.6.self_attn.v_proj.bias]
Loading weights:  25%|▎| 85/338 [00:00<00:00, 4057.82it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  25%|▎| 85/338 [00:00<00:00, 4047.04it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  25%|▌ | 86/338 [00:00<00:00, 4075.08it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  25%|▌ | 86/338 [00:00<00:00, 4064.89it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  26%|█   | 87/338 [00:00<00:00, 4095.54it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  26%|█   | 87/338 [00:00<00:00, 4085.54it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  26%|█   | 88/338 [00:00<00:00, 4110.96it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  26%|█   | 88/338 [00:00<00:00, 4101.10it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  26%|█▌    | 89/338 [00:00<00:00, 4128.94it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  26%|█▌    | 89/338 [00:00<00:00, 4118.96it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  27%|▎| 90/338 [00:00<00:00, 4149.26it/s, Materializing param=model.layers.7.post_attention_layernorm
Loading weights:  27%|▎| 90/338 [00:00<00:00, 4138.80it/s, Materializing param=model.layers.7.post_attention_layernorm
Loading weights:  27%|▊  | 91/338 [00:00<00:00, 4154.63it/s, Materializing param=model.layers.7.self_attn.k_proj.bias]
Loading weights:  27%|▊  | 91/338 [00:00<00:00, 4144.25it/s, Materializing param=model.layers.7.self_attn.k_proj.bias]
Loading weights:  27%|▎| 92/338 [00:00<00:00, 4172.76it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  27%|▎| 92/338 [00:00<00:00, 4162.81it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  28%|▎| 93/338 [00:00<00:00, 4168.53it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  28%|▎| 93/338 [00:00<00:00, 4134.77it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  28%|▊  | 94/338 [00:00<00:00, 4158.65it/s, Materializing param=model.layers.7.self_attn.q_proj.bias]
Loading weights:  28%|▊  | 94/338 [00:00<00:00, 4149.06it/s, Materializing param=model.layers.7.self_attn.q_proj.bias]
Loading weights:  28%|▎| 95/338 [00:00<00:00, 4127.74it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  28%|▎| 95/338 [00:00<00:00, 4118.05it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  28%|▊  | 96/338 [00:00<00:00, 4142.95it/s, Materializing param=model.layers.7.self_attn.v_proj.bias]
Loading weights:  28%|▊  | 96/338 [00:00<00:00, 4133.46it/s, Materializing param=model.layers.7.self_attn.v_proj.bias]
Loading weights:  29%|▎| 97/338 [00:00<00:00, 4161.61it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  29%|▎| 97/338 [00:00<00:00, 4115.56it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  29%|▌ | 98/338 [00:00<00:00, 4140.35it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  29%|▌ | 98/338 [00:00<00:00, 4131.20it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  29%|█▏  | 99/338 [00:00<00:00, 4158.27it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  29%|█▏  | 99/338 [00:00<00:00, 4149.50it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  30%|▉  | 100/338 [00:00<00:00, 4146.33it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  30%|▉  | 100/338 [00:00<00:00, 4136.84it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  30%|█▍   | 101/338 [00:00<00:00, 4162.94it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  30%|█▍   | 101/338 [00:00<00:00, 4029.92it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  30%|▎| 102/338 [00:00<00:00, 4052.16it/s, Materializing param=model.layers.8.post_attention_layernor
Loading weights:  30%|▎| 102/338 [00:00<00:00, 4043.16it/s, Materializing param=model.layers.8.post_attention_layernor
Loading weights:  30%|▌ | 103/338 [00:00<00:00, 4068.57it/s, Materializing param=model.layers.8.self_attn.k_proj.bias]
Loading weights:  30%|▌ | 103/338 [00:00<00:00, 4060.47it/s, Materializing param=model.layers.8.self_attn.k_proj.bias]
Loading weights:  31%|▎| 104/338 [00:00<00:00, 4072.75it/s, Materializing param=model.layers.8.self_attn.k_proj.weight
Loading weights:  31%|▎| 104/338 [00:00<00:00, 4041.84it/s, Materializing param=model.layers.8.self_attn.k_proj.weight
Loading weights:  31%|▎| 105/338 [00:00<00:00, 4029.22it/s, Materializing param=model.layers.8.self_attn.o_proj.weight
Loading weights:  31%|▎| 105/338 [00:00<00:00, 4021.16it/s, Materializing param=model.layers.8.self_attn.o_proj.weight
Loading weights:  31%|▋ | 106/338 [00:00<00:00, 4024.04it/s, Materializing param=model.layers.8.self_attn.q_proj.bias]
Loading weights:  31%|▋ | 106/338 [00:00<00:00, 4011.59it/s, Materializing param=model.layers.8.self_attn.q_proj.bias]
Loading weights:  32%|▎| 107/338 [00:00<00:00, 4031.39it/s, Materializing param=model.layers.8.self_attn.q_proj.weight
Loading weights:  32%|▎| 107/338 [00:00<00:00, 4022.83it/s, Materializing param=model.layers.8.self_attn.q_proj.weight
Loading weights:  32%|▋ | 108/338 [00:00<00:00, 4046.86it/s, Materializing param=model.layers.8.self_attn.v_proj.bias]
Loading weights:  32%|▋ | 108/338 [00:00<00:00, 4038.38it/s, Materializing param=model.layers.8.self_attn.v_proj.bias]
Loading weights:  32%|▎| 109/338 [00:00<00:00, 4058.48it/s, Materializing param=model.layers.8.self_attn.v_proj.weight
Loading weights:  32%|▎| 109/338 [00:00<00:00, 4050.74it/s, Materializing param=model.layers.8.self_attn.v_proj.weight
Loading weights:  33%|▎| 110/338 [00:00<00:00, 4072.57it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  33%|▎| 110/338 [00:00<00:00, 4064.53it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  33%|▉  | 111/338 [00:00<00:00, 4088.27it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  33%|▉  | 111/338 [00:00<00:00, 4080.24it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  33%|▉  | 112/338 [00:00<00:00, 4101.87it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  33%|▉  | 112/338 [00:00<00:00, 4093.75it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  33%|█▋   | 113/338 [00:00<00:00, 4117.39it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  33%|█▋   | 113/338 [00:00<00:00, 4109.53it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  34%|▎| 114/338 [00:00<00:00, 4131.57it/s, Materializing param=model.layers.9.post_attention_layernor
Loading weights:  34%|▎| 114/338 [00:00<00:00, 4094.70it/s, Materializing param=model.layers.9.post_attention_layernor
Loading weights:  34%|▋ | 115/338 [00:00<00:00, 4102.44it/s, Materializing param=model.layers.9.self_attn.k_proj.bias]
Loading weights:  34%|▋ | 115/338 [00:00<00:00, 4094.19it/s, Materializing param=model.layers.9.self_attn.k_proj.bias]
Loading weights:  34%|▎| 116/338 [00:00<00:00, 4116.79it/s, Materializing param=model.layers.9.self_attn.k_proj.weight
Loading weights:  34%|▎| 116/338 [00:00<00:00, 4108.94it/s, Materializing param=model.layers.9.self_attn.k_proj.weight
Loading weights:  35%|▎| 117/338 [00:00<00:00, 4129.57it/s, Materializing param=model.layers.9.self_attn.o_proj.weight
Loading weights:  35%|▎| 117/338 [00:00<00:00, 4121.87it/s, Materializing param=model.layers.9.self_attn.o_proj.weight
Loading weights:  35%|▋ | 118/338 [00:00<00:00, 4144.12it/s, Materializing param=model.layers.9.self_attn.q_proj.bias]
Loading weights:  35%|▋ | 118/338 [00:00<00:00, 4057.65it/s, Materializing param=model.layers.9.self_attn.q_proj.bias]
Loading weights:  35%|▎| 119/338 [00:00<00:00, 4078.96it/s, Materializing param=model.layers.9.self_attn.q_proj.weight
Loading weights:  35%|▎| 119/338 [00:00<00:00, 4071.71it/s, Materializing param=model.layers.9.self_attn.q_proj.weight
Loading weights:  36%|▋ | 120/338 [00:00<00:00, 4085.73it/s, Materializing param=model.layers.9.self_attn.v_proj.bias]
Loading weights:  36%|▋ | 120/338 [00:00<00:00, 4028.40it/s, Materializing param=model.layers.9.self_attn.v_proj.bias]
Loading weights:  36%|▎| 121/338 [00:00<00:00, 4047.97it/s, Materializing param=model.layers.9.self_attn.v_proj.weight
Loading weights:  36%|▎| 121/338 [00:00<00:00, 4040.82it/s, Materializing param=model.layers.9.self_attn.v_proj.weight
Loading weights:  36%|▎| 122/338 [00:00<00:00, 4062.41it/s, Materializing param=model.layers.10.input_layernorm.weight
Loading weights:  36%|▎| 122/338 [00:00<00:00, 4054.97it/s, Materializing param=model.layers.10.input_layernorm.weight
Loading weights:  36%|▋ | 123/338 [00:00<00:00, 4074.62it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  36%|▋ | 123/338 [00:00<00:00, 4067.48it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  37%|▋ | 124/338 [00:00<00:00, 4088.69it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  37%|▋ | 124/338 [00:00<00:00, 4048.37it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  37%|█▍  | 125/338 [00:00<00:00, 4068.41it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  37%|█▍  | 125/338 [00:00<00:00, 4061.57it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  37%|▎| 126/338 [00:00<00:00, 4062.40it/s, Materializing param=model.layers.10.post_attention_layerno
Loading weights:  37%|▎| 126/338 [00:00<00:00, 4054.92it/s, Materializing param=model.layers.10.post_attention_layerno
Loading weights:  38%|▍| 127/338 [00:00<00:00, 4004.30it/s, Materializing param=model.layers.10.self_attn.k_proj.bias]
Loading weights:  38%|▍| 127/338 [00:00<00:00, 3996.79it/s, Materializing param=model.layers.10.self_attn.k_proj.bias]
Loading weights:  38%|▍| 128/338 [00:00<00:00, 3958.29it/s, Materializing param=model.layers.10.self_attn.k_proj.weigh
Loading weights:  38%|▍| 128/338 [00:00<00:00, 3951.44it/s, Materializing param=model.layers.10.self_attn.k_proj.weigh
Loading weights:  38%|▍| 129/338 [00:00<00:00, 3949.06it/s, Materializing param=model.layers.10.self_attn.o_proj.weigh
Loading weights:  38%|▍| 129/338 [00:00<00:00, 3942.56it/s, Materializing param=model.layers.10.self_attn.o_proj.weigh
Loading weights:  38%|▍| 130/338 [00:00<00:00, 3960.89it/s, Materializing param=model.layers.10.self_attn.q_proj.bias]
Loading weights:  38%|▍| 130/338 [00:00<00:00, 3954.39it/s, Materializing param=model.layers.10.self_attn.q_proj.bias]
Loading weights:  39%|▍| 131/338 [00:00<00:00, 3974.41it/s, Materializing param=model.layers.10.self_attn.q_proj.weigh
Loading weights:  39%|▍| 131/338 [00:00<00:00, 3967.92it/s, Materializing param=model.layers.10.self_attn.q_proj.weigh
Loading weights:  39%|▍| 132/338 [00:00<00:00, 3972.16it/s, Materializing param=model.layers.10.self_attn.v_proj.bias]
Loading weights:  39%|▍| 132/338 [00:00<00:00, 3965.62it/s, Materializing param=model.layers.10.self_attn.v_proj.bias]
Loading weights:  39%|▍| 133/338 [00:00<00:00, 3985.22it/s, Materializing param=model.layers.10.self_attn.v_proj.weigh
Loading weights:  39%|▍| 133/338 [00:00<00:00, 3978.88it/s, Materializing param=model.layers.10.self_attn.v_proj.weigh
Loading weights:  40%|▍| 134/338 [00:00<00:00, 3998.33it/s, Materializing param=model.layers.11.input_layernorm.weight
Loading weights:  40%|▍| 134/338 [00:00<00:00, 3992.16it/s, Materializing param=model.layers.11.input_layernorm.weight
Loading weights:  40%|▊ | 135/338 [00:00<00:00, 3985.75it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  40%|▊ | 135/338 [00:00<00:00, 3979.33it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  40%|▊ | 136/338 [00:00<00:00, 3998.16it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  40%|▊ | 136/338 [00:00<00:00, 3991.95it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  41%|█▌  | 137/338 [00:00<00:00, 4009.40it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  41%|█▌  | 137/338 [00:00<00:00, 3997.22it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  41%|▍| 138/338 [00:00<00:00, 3983.17it/s, Materializing param=model.layers.11.post_attention_layerno
Loading weights:  41%|▍| 138/338 [00:00<00:00, 3976.30it/s, Materializing param=model.layers.11.post_attention_layerno
Loading weights:  41%|▍| 139/338 [00:00<00:00, 3992.82it/s, Materializing param=model.layers.11.self_attn.k_proj.bias]
Loading weights:  41%|▍| 139/338 [00:00<00:00, 3979.88it/s, Materializing param=model.layers.11.self_attn.k_proj.bias]
Loading weights:  41%|▍| 140/338 [00:00<00:00, 3987.17it/s, Materializing param=model.layers.11.self_attn.k_proj.weigh
Loading weights:  41%|▍| 140/338 [00:00<00:00, 3971.64it/s, Materializing param=model.layers.11.self_attn.k_proj.weigh
Loading weights:  42%|▍| 141/338 [00:00<00:00, 3927.85it/s, Materializing param=model.layers.11.self_attn.o_proj.weigh
Loading weights:  42%|▍| 141/338 [00:00<00:00, 3921.60it/s, Materializing param=model.layers.11.self_attn.o_proj.weigh
Loading weights:  42%|▍| 142/338 [00:00<00:00, 3939.82it/s, Materializing param=model.layers.11.self_attn.q_proj.bias]
Loading weights:  42%|▍| 142/338 [00:00<00:00, 3933.97it/s, Materializing param=model.layers.11.self_attn.q_proj.bias]
Loading weights:  42%|▍| 143/338 [00:00<00:00, 3930.73it/s, Materializing param=model.layers.11.self_attn.q_proj.weigh
Loading weights:  42%|▍| 143/338 [00:00<00:00, 3924.94it/s, Materializing param=model.layers.11.self_attn.q_proj.weigh
Loading weights:  43%|▍| 144/338 [00:00<00:00, 3916.53it/s, Materializing param=model.layers.11.self_attn.v_proj.bias]
Loading weights:  43%|▍| 144/338 [00:00<00:00, 3910.47it/s, Materializing param=model.layers.11.self_attn.v_proj.bias]
Loading weights:  43%|▍| 145/338 [00:00<00:00, 3928.11it/s, Materializing param=model.layers.11.self_attn.v_proj.weigh
Loading weights:  43%|▍| 145/338 [00:00<00:00, 3922.72it/s, Materializing param=model.layers.11.self_attn.v_proj.weigh
Loading weights:  43%|▍| 146/338 [00:00<00:00, 3940.54it/s, Materializing param=model.layers.12.input_layernorm.weight
Loading weights:  43%|▍| 146/338 [00:00<00:00, 3934.87it/s, Materializing param=model.layers.12.input_layernorm.weight
Loading weights:  43%|▊ | 147/338 [00:00<00:00, 3951.03it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  43%|▊ | 147/338 [00:00<00:00, 3945.24it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  44%|▉ | 148/338 [00:00<00:00, 3962.80it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  44%|▉ | 148/338 [00:00<00:00, 3957.04it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  44%|█▊  | 149/338 [00:00<00:00, 3973.29it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  44%|█▊  | 149/338 [00:00<00:00, 3967.64it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  44%|▍| 150/338 [00:00<00:00, 3985.34it/s, Materializing param=model.layers.12.post_attention_layerno
Loading weights:  44%|▍| 150/338 [00:00<00:00, 3979.82it/s, Materializing param=model.layers.12.post_attention_layerno
Loading weights:  45%|▍| 151/338 [00:00<00:00, 3981.32it/s, Materializing param=model.layers.12.self_attn.k_proj.bias]
Loading weights:  45%|▍| 151/338 [00:00<00:00, 3975.84it/s, Materializing param=model.layers.12.self_attn.k_proj.bias]
Loading weights:  45%|▍| 152/338 [00:00<00:00, 3991.32it/s, Materializing param=model.layers.12.self_attn.k_proj.weigh
Loading weights:  45%|▍| 152/338 [00:00<00:00, 3985.78it/s, Materializing param=model.layers.12.self_attn.k_proj.weigh
Loading weights:  45%|▍| 153/338 [00:00<00:00, 4002.97it/s, Materializing param=model.layers.12.self_attn.o_proj.weigh
Loading weights:  45%|▍| 153/338 [00:00<00:00, 3997.66it/s, Materializing param=model.layers.12.self_attn.o_proj.weigh
Loading weights:  46%|▍| 154/338 [00:00<00:00, 4004.48it/s, Materializing param=model.layers.12.self_attn.q_proj.bias]
Loading weights:  46%|▍| 154/338 [00:00<00:00, 3998.21it/s, Materializing param=model.layers.12.self_attn.q_proj.bias]
Loading weights:  46%|▍| 155/338 [00:00<00:00, 4015.13it/s, Materializing param=model.layers.12.self_attn.q_proj.weigh
Loading weights:  46%|▍| 155/338 [00:00<00:00, 4009.88it/s, Materializing param=model.layers.12.self_attn.q_proj.weigh
Loading weights:  46%|▍| 156/338 [00:00<00:00, 4027.05it/s, Materializing param=model.layers.12.self_attn.v_proj.bias]
Loading weights:  46%|▍| 156/338 [00:00<00:00, 4021.76it/s, Materializing param=model.layers.12.self_attn.v_proj.bias]
Loading weights:  46%|▍| 157/338 [00:00<00:00, 4035.56it/s, Materializing param=model.layers.12.self_attn.v_proj.weigh
Loading weights:  46%|▍| 157/338 [00:00<00:00, 4030.22it/s, Materializing param=model.layers.12.self_attn.v_proj.weigh
Loading weights:  47%|▍| 158/338 [00:00<00:00, 4047.22it/s, Materializing param=model.layers.13.input_layernorm.weight
Loading weights:  47%|▍| 158/338 [00:00<00:00, 4041.77it/s, Materializing param=model.layers.13.input_layernorm.weight
Loading weights:  47%|▉ | 159/338 [00:00<00:00, 4007.01it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  47%|▉ | 159/338 [00:00<00:00, 4001.41it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  47%|▉ | 160/338 [00:00<00:00, 4017.58it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  47%|▉ | 160/338 [00:00<00:00, 4012.42it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  48%|█▉  | 161/338 [00:00<00:00, 4022.58it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  48%|█▉  | 161/338 [00:00<00:00, 4017.03it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  48%|▍| 162/338 [00:00<00:00, 4033.32it/s, Materializing param=model.layers.13.post_attention_layerno
Loading weights:  48%|▍| 162/338 [00:00<00:00, 4028.08it/s, Materializing param=model.layers.13.post_attention_layerno
Loading weights:  48%|▍| 163/338 [00:00<00:00, 4044.25it/s, Materializing param=model.layers.13.self_attn.k_proj.bias]
Loading weights:  48%|▍| 163/338 [00:00<00:00, 4039.08it/s, Materializing param=model.layers.13.self_attn.k_proj.bias]
Loading weights:  49%|▍| 164/338 [00:00<00:00, 4054.00it/s, Materializing param=model.layers.13.self_attn.k_proj.weigh
Loading weights:  49%|▍| 164/338 [00:00<00:00, 4048.82it/s, Materializing param=model.layers.13.self_attn.k_proj.weigh
Loading weights:  49%|▍| 165/338 [00:00<00:00, 4065.06it/s, Materializing param=model.layers.13.self_attn.o_proj.weigh
Loading weights:  49%|▍| 165/338 [00:00<00:00, 4059.91it/s, Materializing param=model.layers.13.self_attn.o_proj.weigh
Loading weights:  49%|▍| 166/338 [00:00<00:00, 4074.79it/s, Materializing param=model.layers.13.self_attn.q_proj.bias]
Loading weights:  49%|▍| 166/338 [00:00<00:00, 4069.66it/s, Materializing param=model.layers.13.self_attn.q_proj.bias]
Loading weights:  49%|▍| 167/338 [00:00<00:00, 4085.77it/s, Materializing param=model.layers.13.self_attn.q_proj.weigh
Loading weights:  49%|▍| 167/338 [00:00<00:00, 4080.61it/s, Materializing param=model.layers.13.self_attn.q_proj.weigh
Loading weights:  50%|▍| 168/338 [00:00<00:00, 4093.50it/s, Materializing param=model.layers.13.self_attn.v_proj.bias]
Loading weights:  50%|▍| 168/338 [00:00<00:00, 4088.28it/s, Materializing param=model.layers.13.self_attn.v_proj.bias]
Loading weights:  50%|▌| 169/338 [00:00<00:00, 4103.97it/s, Materializing param=model.layers.13.self_attn.v_proj.weigh
Loading weights:  50%|▌| 169/338 [00:00<00:00, 4098.84it/s, Materializing param=model.layers.13.self_attn.v_proj.weigh
Loading weights:  50%|▌| 170/338 [00:00<00:00, 4095.55it/s, Materializing param=model.layers.14.input_layernorm.weight
Loading weights:  50%|▌| 170/338 [00:00<00:00, 4090.24it/s, Materializing param=model.layers.14.input_layernorm.weight
Loading weights:  51%|█ | 171/338 [00:00<00:00, 4067.68it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  51%|█ | 171/338 [00:00<00:00, 4062.45it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  51%|█ | 172/338 [00:00<00:00, 4077.78it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  51%|█ | 172/338 [00:00<00:00, 4072.74it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  51%|██  | 173/338 [00:00<00:00, 4062.75it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  51%|██  | 173/338 [00:00<00:00, 4057.00it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  51%|▌| 174/338 [00:00<00:00, 4069.89it/s, Materializing param=model.layers.14.post_attention_layerno
Loading weights:  51%|▌| 174/338 [00:00<00:00, 4064.61it/s, Materializing param=model.layers.14.post_attention_layerno
Loading weights:  52%|▌| 175/338 [00:00<00:00, 4079.61it/s, Materializing param=model.layers.14.self_attn.k_proj.bias]
Loading weights:  52%|▌| 175/338 [00:00<00:00, 4074.67it/s, Materializing param=model.layers.14.self_attn.k_proj.bias]
Loading weights:  52%|▌| 176/338 [00:00<00:00, 4088.58it/s, Materializing param=model.layers.14.self_attn.k_proj.weigh
Loading weights:  52%|▌| 176/338 [00:00<00:00, 4083.65it/s, Materializing param=model.layers.14.self_attn.k_proj.weigh
Loading weights:  52%|▌| 177/338 [00:00<00:00, 4098.62it/s, Materializing param=model.layers.14.self_attn.o_proj.weigh
Loading weights:  52%|▌| 177/338 [00:00<00:00, 4093.72it/s, Materializing param=model.layers.14.self_attn.o_proj.weigh
Loading weights:  53%|▌| 178/338 [00:00<00:00, 4096.70it/s, Materializing param=model.layers.14.self_attn.q_proj.bias]
Loading weights:  53%|▌| 178/338 [00:00<00:00, 4091.67it/s, Materializing param=model.layers.14.self_attn.q_proj.bias]
Loading weights:  53%|▌| 179/338 [00:00<00:00, 4105.00it/s, Materializing param=model.layers.14.self_attn.q_proj.weigh
Loading weights:  53%|▌| 179/338 [00:00<00:00, 4100.12it/s, Materializing param=model.layers.14.self_attn.q_proj.weigh
Loading weights:  53%|▌| 180/338 [00:00<00:00, 4114.80it/s, Materializing param=model.layers.14.self_attn.v_proj.bias]
Loading weights:  53%|▌| 180/338 [00:00<00:00, 4109.82it/s, Materializing param=model.layers.14.self_attn.v_proj.bias]
Loading weights:  54%|▌| 181/338 [00:00<00:00, 4094.34it/s, Materializing param=model.layers.14.self_attn.v_proj.weigh
Loading weights:  54%|▌| 181/338 [00:00<00:00, 4089.20it/s, Materializing param=model.layers.14.self_attn.v_proj.weigh
Loading weights:  54%|▌| 182/338 [00:00<00:00, 4102.38it/s, Materializing param=model.layers.15.input_layernorm.weight
Loading weights:  54%|▌| 182/338 [00:00<00:00, 4097.63it/s, Materializing param=model.layers.15.input_layernorm.weight
Loading weights:  54%|█ | 183/338 [00:00<00:00, 4112.00it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  54%|█ | 183/338 [00:00<00:00, 4107.27it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  54%|█ | 184/338 [00:00<00:00, 4120.54it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  54%|█ | 184/338 [00:00<00:00, 4115.79it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  55%|██▏ | 185/338 [00:00<00:00, 4130.36it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  55%|██▏ | 185/338 [00:00<00:00, 4125.51it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  55%|▌| 186/338 [00:00<00:00, 4107.45it/s, Materializing param=model.layers.15.post_attention_layerno
Loading weights:  55%|▌| 186/338 [00:00<00:00, 4102.46it/s, Materializing param=model.layers.15.post_attention_layerno
Loading weights:  55%|▌| 187/338 [00:00<00:00, 4097.05it/s, Materializing param=model.layers.15.self_attn.k_proj.bias]
Loading weights:  55%|▌| 187/338 [00:00<00:00, 4092.24it/s, Materializing param=model.layers.15.self_attn.k_proj.bias]
Loading weights:  56%|▌| 188/338 [00:00<00:00, 4104.93it/s, Materializing param=model.layers.15.self_attn.k_proj.weigh
Loading weights:  56%|▌| 188/338 [00:00<00:00, 4100.30it/s, Materializing param=model.layers.15.self_attn.k_proj.weigh
Loading weights:  56%|▌| 189/338 [00:00<00:00, 4114.47it/s, Materializing param=model.layers.15.self_attn.o_proj.weigh
Loading weights:  56%|▌| 189/338 [00:00<00:00, 4109.82it/s, Materializing param=model.layers.15.self_attn.o_proj.weigh
Loading weights:  56%|▌| 190/338 [00:00<00:00, 4122.57it/s, Materializing param=model.layers.15.self_attn.q_proj.bias]
Loading weights:  56%|▌| 190/338 [00:00<00:00, 4117.91it/s, Materializing param=model.layers.15.self_attn.q_proj.bias]
Loading weights:  57%|▌| 191/338 [00:00<00:00, 4131.96it/s, Materializing param=model.layers.15.self_attn.q_proj.weigh
Loading weights:  57%|▌| 191/338 [00:00<00:00, 4127.38it/s, Materializing param=model.layers.15.self_attn.q_proj.weigh
Loading weights:  57%|▌| 192/338 [00:00<00:00, 4128.93it/s, Materializing param=model.layers.15.self_attn.v_proj.bias]
Loading weights:  57%|▌| 192/338 [00:00<00:00, 4123.85it/s, Materializing param=model.layers.15.self_attn.v_proj.bias]
Loading weights:  57%|▌| 193/338 [00:00<00:00, 4137.75it/s, Materializing param=model.layers.15.self_attn.v_proj.weigh
Loading weights:  57%|▌| 193/338 [00:00<00:00, 4133.23it/s, Materializing param=model.layers.15.self_attn.v_proj.weigh
Loading weights:  57%|▌| 194/338 [00:00<00:00, 4147.23it/s, Materializing param=model.layers.16.input_layernorm.weight
Loading weights:  57%|▌| 194/338 [00:00<00:00, 4142.78it/s, Materializing param=model.layers.16.input_layernorm.weight
Loading weights:  58%|█▏| 195/338 [00:00<00:00, 4095.51it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  58%|█▏| 195/338 [00:00<00:00, 4090.57it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  58%|█▏| 196/338 [00:00<00:00, 4103.81it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  58%|█▏| 196/338 [00:00<00:00, 4099.21it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  58%|██▎ | 197/338 [00:00<00:00, 4112.53it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  58%|██▎ | 197/338 [00:00<00:00, 4108.04it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  59%|▌| 198/338 [00:00<00:00, 4086.10it/s, Materializing param=model.layers.16.post_attention_layerno
Loading weights:  59%|▌| 198/338 [00:00<00:00, 4081.25it/s, Materializing param=model.layers.16.post_attention_layerno
Loading weights:  59%|▌| 199/338 [00:00<00:00, 4094.25it/s, Materializing param=model.layers.16.self_attn.k_proj.bias]
Loading weights:  59%|▌| 199/338 [00:00<00:00, 4089.88it/s, Materializing param=model.layers.16.self_attn.k_proj.bias]
Loading weights:  59%|▌| 200/338 [00:00<00:00, 4103.23it/s, Materializing param=model.layers.16.self_attn.k_proj.weigh
Loading weights:  59%|▌| 200/338 [00:00<00:00, 4098.84it/s, Materializing param=model.layers.16.self_attn.k_proj.weigh
Loading weights:  59%|▌| 201/338 [00:00<00:00, 4109.50it/s, Materializing param=model.layers.16.self_attn.o_proj.weigh
Loading weights:  59%|▌| 201/338 [00:00<00:00, 4105.03it/s, Materializing param=model.layers.16.self_attn.o_proj.weigh
Loading weights:  60%|▌| 202/338 [00:00<00:00, 4118.12it/s, Materializing param=model.layers.16.self_attn.q_proj.bias]
Loading weights:  60%|▌| 202/338 [00:00<00:00, 4113.70it/s, Materializing param=model.layers.16.self_attn.q_proj.bias]
Loading weights:  60%|▌| 203/338 [00:00<00:00, 4120.34it/s, Materializing param=model.layers.16.self_attn.q_proj.weigh
Loading weights:  60%|▌| 203/338 [00:00<00:00, 4115.76it/s, Materializing param=model.layers.16.self_attn.q_proj.weigh
Loading weights:  60%|▌| 204/338 [00:00<00:00, 4127.55it/s, Materializing param=model.layers.16.self_attn.v_proj.bias]
Loading weights:  60%|▌| 204/338 [00:00<00:00, 4123.32it/s, Materializing param=model.layers.16.self_attn.v_proj.bias]
Loading weights:  61%|▌| 205/338 [00:00<00:00, 4136.37it/s, Materializing param=model.layers.16.self_attn.v_proj.weigh
Loading weights:  61%|▌| 205/338 [00:00<00:00, 4131.86it/s, Materializing param=model.layers.16.self_attn.v_proj.weigh
Loading weights:  61%|▌| 206/338 [00:00<00:00, 4127.66it/s, Materializing param=model.layers.17.input_layernorm.weight
Loading weights:  61%|▌| 206/338 [00:00<00:00, 4123.25it/s, Materializing param=model.layers.17.input_layernorm.weight
Loading weights:  61%|█▏| 207/338 [00:00<00:00, 4134.82it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  61%|█▏| 207/338 [00:00<00:00, 4130.63it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  62%|█▏| 208/338 [00:00<00:00, 4143.62it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  62%|█▏| 208/338 [00:00<00:00, 4139.36it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  62%|██▍ | 209/338 [00:00<00:00, 4144.12it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  62%|██▍ | 209/338 [00:00<00:00, 4138.09it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  62%|▌| 210/338 [00:00<00:00, 4149.35it/s, Materializing param=model.layers.17.post_attention_layerno
Loading weights:  62%|▌| 210/338 [00:00<00:00, 4144.84it/s, Materializing param=model.layers.17.post_attention_layerno
Loading weights:  62%|▌| 211/338 [00:00<00:00, 4156.87it/s, Materializing param=model.layers.17.self_attn.k_proj.bias]
Loading weights:  62%|▌| 211/338 [00:00<00:00, 4152.72it/s, Materializing param=model.layers.17.self_attn.k_proj.bias]
Loading weights:  63%|▋| 212/338 [00:00<00:00, 4165.40it/s, Materializing param=model.layers.17.self_attn.k_proj.weigh
Loading weights:  63%|▋| 212/338 [00:00<00:00, 4161.13it/s, Materializing param=model.layers.17.self_attn.k_proj.weigh
Loading weights:  63%|▋| 213/338 [00:00<00:00, 4172.75it/s, Materializing param=model.layers.17.self_attn.o_proj.weigh
Loading weights:  63%|▋| 213/338 [00:00<00:00, 4168.53it/s, Materializing param=model.layers.17.self_attn.o_proj.weigh
Loading weights:  63%|▋| 214/338 [00:00<00:00, 4181.15it/s, Materializing param=model.layers.17.self_attn.q_proj.bias]
Loading weights:  63%|▋| 214/338 [00:00<00:00, 4176.91it/s, Materializing param=model.layers.17.self_attn.q_proj.bias]
Loading weights:  64%|▋| 215/338 [00:00<00:00, 4187.18it/s, Materializing param=model.layers.17.self_attn.q_proj.weigh
Loading weights:  64%|▋| 215/338 [00:00<00:00, 4182.94it/s, Materializing param=model.layers.17.self_attn.q_proj.weigh
Loading weights:  64%|▋| 216/338 [00:00<00:00, 4195.59it/s, Materializing param=model.layers.17.self_attn.v_proj.bias]
Loading weights:  64%|▋| 216/338 [00:00<00:00, 4191.34it/s, Materializing param=model.layers.17.self_attn.v_proj.bias]
Loading weights:  64%|▋| 217/338 [00:00<00:00, 4190.83it/s, Materializing param=model.layers.17.self_attn.v_proj.weigh
Loading weights:  64%|▋| 217/338 [00:00<00:00, 4186.55it/s, Materializing param=model.layers.17.self_attn.v_proj.weigh
Loading weights:  64%|▋| 218/338 [00:00<00:00, 4197.64it/s, Materializing param=model.layers.18.input_layernorm.weight
Loading weights:  64%|▋| 218/338 [00:00<00:00, 4193.50it/s, Materializing param=model.layers.18.input_layernorm.weight
Loading weights:  65%|█▎| 219/338 [00:00<00:00, 4205.94it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  65%|█▎| 219/338 [00:00<00:00, 4201.61it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  65%|█▎| 220/338 [00:00<00:00, 4177.16it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  65%|█▎| 220/338 [00:00<00:00, 4172.68it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  65%|██▌ | 221/338 [00:00<00:00, 4150.99it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  65%|██▌ | 221/338 [00:00<00:00, 4146.76it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  66%|▋| 222/338 [00:00<00:00, 4150.80it/s, Materializing param=model.layers.18.post_attention_layerno
Loading weights:  66%|▋| 222/338 [00:00<00:00, 4146.45it/s, Materializing param=model.layers.18.post_attention_layerno
Loading weights:  66%|▋| 223/338 [00:00<00:00, 4148.85it/s, Materializing param=model.layers.18.self_attn.k_proj.bias]
Loading weights:  66%|▋| 223/338 [00:00<00:00, 4144.50it/s, Materializing param=model.layers.18.self_attn.k_proj.bias]
Loading weights:  66%|▋| 224/338 [00:00<00:00, 4155.09it/s, Materializing param=model.layers.18.self_attn.k_proj.weigh
Loading weights:  66%|▋| 224/338 [00:00<00:00, 4151.11it/s, Materializing param=model.layers.18.self_attn.k_proj.weigh
Loading weights:  67%|▋| 225/338 [00:00<00:00, 4162.74it/s, Materializing param=model.layers.18.self_attn.o_proj.weigh
Loading weights:  67%|▋| 225/338 [00:00<00:00, 4158.60it/s, Materializing param=model.layers.18.self_attn.o_proj.weigh
Loading weights:  67%|▋| 226/338 [00:00<00:00, 4167.80it/s, Materializing param=model.layers.18.self_attn.q_proj.bias]
Loading weights:  67%|▋| 226/338 [00:00<00:00, 4163.76it/s, Materializing param=model.layers.18.self_attn.q_proj.bias]
Loading weights:  67%|▋| 227/338 [00:00<00:00, 4174.96it/s, Materializing param=model.layers.18.self_attn.q_proj.weigh
Loading weights:  67%|▋| 227/338 [00:00<00:00, 4169.40it/s, Materializing param=model.layers.18.self_attn.q_proj.weigh
Loading weights:  67%|▋| 228/338 [00:00<00:00, 4179.02it/s, Materializing param=model.layers.18.self_attn.v_proj.bias]
Loading weights:  67%|▋| 228/338 [00:00<00:00, 4175.02it/s, Materializing param=model.layers.18.self_attn.v_proj.bias]
Loading weights:  68%|▋| 229/338 [00:00<00:00, 4186.90it/s, Materializing param=model.layers.18.self_attn.v_proj.weigh
Loading weights:  68%|▋| 229/338 [00:00<00:00, 4183.03it/s, Materializing param=model.layers.18.self_attn.v_proj.weigh
Loading weights:  68%|▋| 230/338 [00:00<00:00, 4193.81it/s, Materializing param=model.layers.19.input_layernorm.weight
Loading weights:  68%|▋| 230/338 [00:00<00:00, 4189.99it/s, Materializing param=model.layers.19.input_layernorm.weight
Loading weights:  68%|█▎| 231/338 [00:00<00:00, 4201.85it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  68%|█▎| 231/338 [00:00<00:00, 4197.99it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  69%|█▎| 232/338 [00:00<00:00, 4198.34it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  69%|█▎| 232/338 [00:00<00:00, 4194.27it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  69%|██▊ | 233/338 [00:00<00:00, 4205.87it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  69%|██▊ | 233/338 [00:00<00:00, 4201.91it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  69%|▋| 234/338 [00:00<00:00, 4213.54it/s, Materializing param=model.layers.19.post_attention_layerno
Loading weights:  69%|▋| 234/338 [00:00<00:00, 4209.50it/s, Materializing param=model.layers.19.post_attention_layerno
Loading weights:  70%|▋| 235/338 [00:00<00:00, 4219.96it/s, Materializing param=model.layers.19.self_attn.k_proj.bias]
Loading weights:  70%|▋| 235/338 [00:00<00:00, 4215.94it/s, Materializing param=model.layers.19.self_attn.k_proj.bias]
Loading weights:  70%|▋| 236/338 [00:00<00:00, 4227.53it/s, Materializing param=model.layers.19.self_attn.k_proj.weigh
Loading weights:  70%|▋| 236/338 [00:00<00:00, 4223.31it/s, Materializing param=model.layers.19.self_attn.k_proj.weigh
Loading weights:  70%|▋| 237/338 [00:00<00:00, 4222.87it/s, Materializing param=model.layers.19.self_attn.o_proj.weigh
Loading weights:  70%|▋| 237/338 [00:00<00:00, 4218.80it/s, Materializing param=model.layers.19.self_attn.o_proj.weigh
Loading weights:  70%|▋| 238/338 [00:00<00:00, 4228.86it/s, Materializing param=model.layers.19.self_attn.q_proj.bias]
Loading weights:  70%|▋| 238/338 [00:00<00:00, 4224.96it/s, Materializing param=model.layers.19.self_attn.q_proj.bias]
Loading weights:  71%|▋| 239/338 [00:00<00:00, 4236.26it/s, Materializing param=model.layers.19.self_attn.q_proj.weigh
Loading weights:  71%|▋| 239/338 [00:00<00:00, 4232.41it/s, Materializing param=model.layers.19.self_attn.q_proj.weigh
Loading weights:  71%|▋| 240/338 [00:00<00:00, 4224.97it/s, Materializing param=model.layers.19.self_attn.v_proj.bias]
Loading weights:  71%|▋| 240/338 [00:00<00:00, 4221.09it/s, Materializing param=model.layers.19.self_attn.v_proj.bias]
Loading weights:  71%|▋| 241/338 [00:00<00:00, 4232.98it/s, Materializing param=model.layers.19.self_attn.v_proj.weigh
Loading weights:  71%|▋| 241/338 [00:00<00:00, 4229.35it/s, Materializing param=model.layers.19.self_attn.v_proj.weigh
Loading weights:  72%|▋| 242/338 [00:00<00:00, 4241.33it/s, Materializing param=model.layers.20.input_layernorm.weight
Loading weights:  72%|▋| 242/338 [00:00<00:00, 4237.66it/s, Materializing param=model.layers.20.input_layernorm.weight
Loading weights:  72%|█▍| 243/338 [00:00<00:00, 4249.80it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  72%|█▍| 243/338 [00:00<00:00, 4246.15it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  72%|█▍| 244/338 [00:00<00:00, 4258.14it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  72%|█▍| 244/338 [00:00<00:00, 4254.53it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  72%|██▉ | 245/338 [00:00<00:00, 4266.38it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  72%|██▉ | 245/338 [00:00<00:00, 4262.70it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  73%|▋| 246/338 [00:00<00:00, 4274.64it/s, Materializing param=model.layers.20.post_attention_layerno
Loading weights:  73%|▋| 246/338 [00:00<00:00, 4270.97it/s, Materializing param=model.layers.20.post_attention_layerno
Loading weights:  73%|▋| 247/338 [00:00<00:00, 4282.72it/s, Materializing param=model.layers.20.self_attn.k_proj.bias]
Loading weights:  73%|▋| 247/338 [00:00<00:00, 4279.02it/s, Materializing param=model.layers.20.self_attn.k_proj.bias]
Loading weights:  73%|▋| 248/338 [00:00<00:00, 4290.87it/s, Materializing param=model.layers.20.self_attn.k_proj.weigh
Loading weights:  73%|▋| 248/338 [00:00<00:00, 4287.24it/s, Materializing param=model.layers.20.self_attn.k_proj.weigh
Loading weights:  74%|▋| 249/338 [00:00<00:00, 4299.00it/s, Materializing param=model.layers.20.self_attn.o_proj.weigh
Loading weights:  74%|▋| 249/338 [00:00<00:00, 4295.32it/s, Materializing param=model.layers.20.self_attn.o_proj.weigh
Loading weights:  74%|▋| 250/338 [00:00<00:00, 4306.94it/s, Materializing param=model.layers.20.self_attn.q_proj.bias]
Loading weights:  74%|▋| 250/338 [00:00<00:00, 4303.40it/s, Materializing param=model.layers.20.self_attn.q_proj.bias]
Loading weights:  74%|▋| 251/338 [00:00<00:00, 4315.13it/s, Materializing param=model.layers.20.self_attn.q_proj.weigh
Loading weights:  74%|▋| 251/338 [00:00<00:00, 4311.52it/s, Materializing param=model.layers.20.self_attn.q_proj.weigh
Loading weights:  75%|▋| 252/338 [00:00<00:00, 4323.30it/s, Materializing param=model.layers.20.self_attn.v_proj.bias]
Loading weights:  75%|▋| 252/338 [00:00<00:00, 4319.68it/s, Materializing param=model.layers.20.self_attn.v_proj.bias]
Loading weights:  75%|▋| 253/338 [00:00<00:00, 4329.00it/s, Materializing param=model.layers.20.self_attn.v_proj.weigh
Loading weights:  75%|▋| 253/338 [00:00<00:00, 4324.61it/s, Materializing param=model.layers.20.self_attn.v_proj.weigh
Loading weights:  75%|▊| 254/338 [00:00<00:00, 4335.96it/s, Materializing param=model.layers.21.input_layernorm.weight
Loading weights:  75%|▊| 254/338 [00:00<00:00, 4331.95it/s, Materializing param=model.layers.21.input_layernorm.weight
Loading weights:  75%|█▌| 255/338 [00:00<00:00, 4343.27it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  75%|█▌| 255/338 [00:00<00:00, 4339.55it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  76%|█▌| 256/338 [00:00<00:00, 4350.83it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  76%|█▌| 256/338 [00:00<00:00, 4347.01it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  76%|███ | 257/338 [00:00<00:00, 4358.27it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  76%|███ | 257/338 [00:00<00:00, 4354.58it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  76%|▊| 258/338 [00:00<00:00, 4365.96it/s, Materializing param=model.layers.21.post_attention_layerno
Loading weights:  76%|▊| 258/338 [00:00<00:00, 4362.13it/s, Materializing param=model.layers.21.post_attention_layerno
Loading weights:  77%|▊| 259/338 [00:00<00:00, 4373.32it/s, Materializing param=model.layers.21.self_attn.k_proj.bias]
Loading weights:  77%|▊| 259/338 [00:00<00:00, 4369.59it/s, Materializing param=model.layers.21.self_attn.k_proj.bias]
Loading weights:  77%|▊| 260/338 [00:00<00:00, 4380.83it/s, Materializing param=model.layers.21.self_attn.k_proj.weigh
Loading weights:  77%|▊| 260/338 [00:00<00:00, 4377.19it/s, Materializing param=model.layers.21.self_attn.k_proj.weigh
Loading weights:  77%|▊| 261/338 [00:00<00:00, 4388.37it/s, Materializing param=model.layers.21.self_attn.o_proj.weigh
Loading weights:  77%|▊| 261/338 [00:00<00:00, 4384.78it/s, Materializing param=model.layers.21.self_attn.o_proj.weigh
Loading weights:  78%|▊| 262/338 [00:00<00:00, 4396.11it/s, Materializing param=model.layers.21.self_attn.q_proj.bias]
Loading weights:  78%|▊| 262/338 [00:00<00:00, 4392.43it/s, Materializing param=model.layers.21.self_attn.q_proj.bias]
Loading weights:  78%|▊| 263/338 [00:00<00:00, 4403.72it/s, Materializing param=model.layers.21.self_attn.q_proj.weigh
Loading weights:  78%|▊| 263/338 [00:00<00:00, 4400.14it/s, Materializing param=model.layers.21.self_attn.q_proj.weigh
Loading weights:  78%|▊| 264/338 [00:00<00:00, 4411.36it/s, Materializing param=model.layers.21.self_attn.v_proj.bias]
Loading weights:  78%|▊| 264/338 [00:00<00:00, 4407.73it/s, Materializing param=model.layers.21.self_attn.v_proj.bias]
Loading weights:  78%|▊| 265/338 [00:00<00:00, 4418.90it/s, Materializing param=model.layers.21.self_attn.v_proj.weigh
Loading weights:  78%|▊| 265/338 [00:00<00:00, 4415.32it/s, Materializing param=model.layers.21.self_attn.v_proj.weigh
Loading weights:  79%|▊| 266/338 [00:00<00:00, 4426.60it/s, Materializing param=model.layers.22.input_layernorm.weight
Loading weights:  79%|▊| 266/338 [00:00<00:00, 4422.93it/s, Materializing param=model.layers.22.input_layernorm.weight
Loading weights:  79%|█▌| 267/338 [00:00<00:00, 4434.01it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  79%|█▌| 267/338 [00:00<00:00, 4430.34it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  79%|█▌| 268/338 [00:00<00:00, 4441.38it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  79%|█▌| 268/338 [00:00<00:00, 4437.70it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  80%|███▏| 269/338 [00:00<00:00, 4448.79it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  80%|███▏| 269/338 [00:00<00:00, 4445.11it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  80%|▊| 270/338 [00:00<00:00, 4455.64it/s, Materializing param=model.layers.22.post_attention_layerno
Loading weights:  80%|▊| 270/338 [00:00<00:00, 4451.85it/s, Materializing param=model.layers.22.post_attention_layerno
Loading weights:  80%|▊| 271/338 [00:00<00:00, 4462.80it/s, Materializing param=model.layers.22.self_attn.k_proj.bias]
Loading weights:  80%|▊| 271/338 [00:00<00:00, 4459.15it/s, Materializing param=model.layers.22.self_attn.k_proj.bias]
Loading weights:  80%|▊| 272/338 [00:00<00:00, 4470.05it/s, Materializing param=model.layers.22.self_attn.k_proj.weigh
Loading weights:  80%|▊| 272/338 [00:00<00:00, 4466.34it/s, Materializing param=model.layers.22.self_attn.k_proj.weigh
Loading weights:  81%|▊| 273/338 [00:00<00:00, 4477.19it/s, Materializing param=model.layers.22.self_attn.o_proj.weigh
Loading weights:  81%|▊| 273/338 [00:00<00:00, 4473.58it/s, Materializing param=model.layers.22.self_attn.o_proj.weigh
Loading weights:  81%|▊| 274/338 [00:00<00:00, 4484.52it/s, Materializing param=model.layers.22.self_attn.q_proj.bias]
Loading weights:  81%|▊| 274/338 [00:00<00:00, 4480.83it/s, Materializing param=model.layers.22.self_attn.q_proj.bias]
Loading weights:  81%|▊| 275/338 [00:00<00:00, 4491.72it/s, Materializing param=model.layers.22.self_attn.q_proj.weigh
Loading weights:  81%|▊| 275/338 [00:00<00:00, 4487.96it/s, Materializing param=model.layers.22.self_attn.q_proj.weigh
Loading weights:  82%|▊| 276/338 [00:00<00:00, 4498.75it/s, Materializing param=model.layers.22.self_attn.v_proj.bias]
Loading weights:  82%|▊| 276/338 [00:00<00:00, 4495.24it/s, Materializing param=model.layers.22.self_attn.v_proj.bias]
Loading weights:  82%|▊| 277/338 [00:00<00:00, 4506.10it/s, Materializing param=model.layers.22.self_attn.v_proj.weigh
Loading weights:  82%|▊| 277/338 [00:00<00:00, 4502.54it/s, Materializing param=model.layers.22.self_attn.v_proj.weigh
Loading weights:  82%|▊| 278/338 [00:00<00:00, 4513.39it/s, Materializing param=model.layers.23.input_layernorm.weight
Loading weights:  82%|▊| 278/338 [00:00<00:00, 4509.62it/s, Materializing param=model.layers.23.input_layernorm.weight
Loading weights:  83%|█▋| 279/338 [00:00<00:00, 4520.23it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  83%|█▋| 279/338 [00:00<00:00, 4516.43it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  83%|█▋| 280/338 [00:00<00:00, 4526.99it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  83%|█▋| 280/338 [00:00<00:00, 4523.31it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  83%|███▎| 281/338 [00:00<00:00, 4533.82it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  83%|███▎| 281/338 [00:00<00:00, 4530.13it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  83%|▊| 282/338 [00:00<00:00, 4540.81it/s, Materializing param=model.layers.23.post_attention_layerno
Loading weights:  83%|▊| 282/338 [00:00<00:00, 4537.10it/s, Materializing param=model.layers.23.post_attention_layerno
Loading weights:  84%|▊| 283/338 [00:00<00:00, 4547.66it/s, Materializing param=model.layers.23.self_attn.k_proj.bias]
Loading weights:  84%|▊| 283/338 [00:00<00:00, 4544.05it/s, Materializing param=model.layers.23.self_attn.k_proj.bias]
Loading weights:  84%|▊| 284/338 [00:00<00:00, 4554.69it/s, Materializing param=model.layers.23.self_attn.k_proj.weigh
Loading weights:  84%|▊| 284/338 [00:00<00:00, 4549.22it/s, Materializing param=model.layers.23.self_attn.k_proj.weigh
Loading weights:  84%|▊| 285/338 [00:00<00:00, 4558.54it/s, Materializing param=model.layers.23.self_attn.o_proj.weigh
Loading weights:  84%|▊| 285/338 [00:00<00:00, 4554.93it/s, Materializing param=model.layers.23.self_attn.o_proj.weigh
Loading weights:  85%|▊| 286/338 [00:00<00:00, 4565.45it/s, Materializing param=model.layers.23.self_attn.q_proj.bias]
Loading weights:  85%|▊| 286/338 [00:00<00:00, 4561.71it/s, Materializing param=model.layers.23.self_attn.q_proj.bias]
Loading weights:  85%|▊| 287/338 [00:00<00:00, 4572.10it/s, Materializing param=model.layers.23.self_attn.q_proj.weigh
Loading weights:  85%|▊| 287/338 [00:00<00:00, 4568.51it/s, Materializing param=model.layers.23.self_attn.q_proj.weigh
Loading weights:  85%|▊| 288/338 [00:00<00:00, 4578.99it/s, Materializing param=model.layers.23.self_attn.v_proj.bias]
Loading weights:  85%|▊| 288/338 [00:00<00:00, 4575.31it/s, Materializing param=model.layers.23.self_attn.v_proj.bias]
Loading weights:  86%|▊| 289/338 [00:00<00:00, 4585.60it/s, Materializing param=model.layers.23.self_attn.v_proj.weigh
Loading weights:  86%|▊| 289/338 [00:00<00:00, 4581.93it/s, Materializing param=model.layers.23.self_attn.v_proj.weigh
Loading weights:  86%|▊| 290/338 [00:00<00:00, 4592.38it/s, Materializing param=model.layers.24.input_layernorm.weight
Loading weights:  86%|▊| 290/338 [00:00<00:00, 4588.78it/s, Materializing param=model.layers.24.input_layernorm.weight
Loading weights:  86%|█▋| 291/338 [00:00<00:00, 4599.21it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  86%|█▋| 291/338 [00:00<00:00, 4595.62it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  86%|█▋| 292/338 [00:00<00:00, 4605.83it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  86%|█▋| 292/338 [00:00<00:00, 4602.11it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  87%|███▍| 293/338 [00:00<00:00, 4612.33it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  87%|███▍| 293/338 [00:00<00:00, 4608.71it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  87%|▊| 294/338 [00:00<00:00, 4619.14it/s, Materializing param=model.layers.24.post_attention_layerno
Loading weights:  87%|▊| 294/338 [00:00<00:00, 4615.44it/s, Materializing param=model.layers.24.post_attention_layerno
Loading weights:  87%|▊| 295/338 [00:00<00:00, 4625.70it/s, Materializing param=model.layers.24.self_attn.k_proj.bias]
Loading weights:  87%|▊| 295/338 [00:00<00:00, 4622.23it/s, Materializing param=model.layers.24.self_attn.k_proj.bias]
Loading weights:  88%|▉| 296/338 [00:00<00:00, 4632.62it/s, Materializing param=model.layers.24.self_attn.k_proj.weigh
Loading weights:  88%|▉| 296/338 [00:00<00:00, 4629.04it/s, Materializing param=model.layers.24.self_attn.k_proj.weigh
Loading weights:  88%|▉| 297/338 [00:00<00:00, 4639.41it/s, Materializing param=model.layers.24.self_attn.o_proj.weigh
Loading weights:  88%|▉| 297/338 [00:00<00:00, 4635.94it/s, Materializing param=model.layers.24.self_attn.o_proj.weigh
Loading weights:  88%|▉| 298/338 [00:00<00:00, 4646.20it/s, Materializing param=model.layers.24.self_attn.q_proj.bias]
Loading weights:  88%|▉| 298/338 [00:00<00:00, 4642.61it/s, Materializing param=model.layers.24.self_attn.q_proj.bias]
Loading weights:  88%|▉| 299/338 [00:00<00:00, 4652.76it/s, Materializing param=model.layers.24.self_attn.q_proj.weigh
Loading weights:  88%|▉| 299/338 [00:00<00:00, 4649.18it/s, Materializing param=model.layers.24.self_attn.q_proj.weigh
Loading weights:  89%|▉| 300/338 [00:00<00:00, 4659.39it/s, Materializing param=model.layers.24.self_attn.v_proj.bias]
Loading weights:  89%|▉| 300/338 [00:00<00:00, 4655.84it/s, Materializing param=model.layers.24.self_attn.v_proj.bias]
Loading weights:  89%|▉| 301/338 [00:00<00:00, 4666.00it/s, Materializing param=model.layers.24.self_attn.v_proj.weigh
Loading weights:  89%|▉| 301/338 [00:00<00:00, 4662.44it/s, Materializing param=model.layers.24.self_attn.v_proj.weigh
Loading weights:  89%|▉| 302/338 [00:00<00:00, 4672.54it/s, Materializing param=model.layers.25.input_layernorm.weight
Loading weights:  89%|▉| 302/338 [00:00<00:00, 4668.94it/s, Materializing param=model.layers.25.input_layernorm.weight
Loading weights:  90%|█▊| 303/338 [00:00<00:00, 4679.16it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  90%|█▊| 303/338 [00:00<00:00, 4675.37it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  90%|█▊| 304/338 [00:00<00:00, 4685.39it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  90%|█▊| 304/338 [00:00<00:00, 4681.74it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  90%|███▌| 305/338 [00:00<00:00, 4691.75it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  90%|███▌| 305/338 [00:00<00:00, 4688.25it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  91%|▉| 306/338 [00:00<00:00, 4698.33it/s, Materializing param=model.layers.25.post_attention_layerno
Loading weights:  91%|▉| 306/338 [00:00<00:00, 4694.69it/s, Materializing param=model.layers.25.post_attention_layerno
Loading weights:  91%|▉| 307/338 [00:00<00:00, 4704.66it/s, Materializing param=model.layers.25.self_attn.k_proj.bias]
Loading weights:  91%|▉| 307/338 [00:00<00:00, 4701.09it/s, Materializing param=model.layers.25.self_attn.k_proj.bias]
Loading weights:  91%|▉| 308/338 [00:00<00:00, 4711.00it/s, Materializing param=model.layers.25.self_attn.k_proj.weigh
Loading weights:  91%|▉| 308/338 [00:00<00:00, 4707.50it/s, Materializing param=model.layers.25.self_attn.k_proj.weigh
Loading weights:  91%|▉| 309/338 [00:00<00:00, 4717.30it/s, Materializing param=model.layers.25.self_attn.o_proj.weigh
Loading weights:  91%|▉| 309/338 [00:00<00:00, 4713.76it/s, Materializing param=model.layers.25.self_attn.o_proj.weigh
Loading weights:  92%|▉| 310/338 [00:00<00:00, 4723.73it/s, Materializing param=model.layers.25.self_attn.q_proj.bias]
Loading weights:  92%|▉| 310/338 [00:00<00:00, 4720.21it/s, Materializing param=model.layers.25.self_attn.q_proj.bias]
Loading weights:  92%|▉| 311/338 [00:00<00:00, 4730.01it/s, Materializing param=model.layers.25.self_attn.q_proj.weigh
Loading weights:  92%|▉| 311/338 [00:00<00:00, 4726.48it/s, Materializing param=model.layers.25.self_attn.q_proj.weigh
Loading weights:  92%|▉| 312/338 [00:00<00:00, 4736.34it/s, Materializing param=model.layers.25.self_attn.v_proj.bias]
Loading weights:  92%|▉| 312/338 [00:00<00:00, 4732.80it/s, Materializing param=model.layers.25.self_attn.v_proj.bias]
Loading weights:  93%|▉| 313/338 [00:00<00:00, 4742.60it/s, Materializing param=model.layers.25.self_attn.v_proj.weigh
Loading weights:  93%|▉| 313/338 [00:00<00:00, 4739.10it/s, Materializing param=model.layers.25.self_attn.v_proj.weigh
Loading weights:  93%|▉| 314/338 [00:00<00:00, 4748.93it/s, Materializing param=model.layers.26.input_layernorm.weight
Loading weights:  93%|▉| 314/338 [00:00<00:00, 4745.41it/s, Materializing param=model.layers.26.input_layernorm.weight
Loading weights:  93%|█▊| 315/338 [00:00<00:00, 4755.22it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  93%|█▊| 315/338 [00:00<00:00, 4751.74it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  93%|█▊| 316/338 [00:00<00:00, 4761.07it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  93%|█▊| 316/338 [00:00<00:00, 4755.91it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  94%|███▊| 317/338 [00:00<00:00, 4763.56it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  94%|███▊| 317/338 [00:00<00:00, 4759.62it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  94%|▉| 318/338 [00:00<00:00, 4769.24it/s, Materializing param=model.layers.26.post_attention_layerno
Loading weights:  94%|▉| 318/338 [00:00<00:00, 4765.45it/s, Materializing param=model.layers.26.post_attention_layerno
Loading weights:  94%|▉| 319/338 [00:00<00:00, 4774.86it/s, Materializing param=model.layers.26.self_attn.k_proj.bias]
Loading weights:  94%|▉| 319/338 [00:00<00:00, 4771.32it/s, Materializing param=model.layers.26.self_attn.k_proj.bias]
Loading weights:  95%|▉| 320/338 [00:00<00:00, 4781.01it/s, Materializing param=model.layers.26.self_attn.k_proj.weigh
Loading weights:  95%|▉| 320/338 [00:00<00:00, 4777.37it/s, Materializing param=model.layers.26.self_attn.k_proj.weigh
Loading weights:  95%|▉| 321/338 [00:00<00:00, 4786.98it/s, Materializing param=model.layers.26.self_attn.o_proj.weigh
Loading weights:  95%|▉| 321/338 [00:00<00:00, 4783.53it/s, Materializing param=model.layers.26.self_attn.o_proj.weigh
Loading weights:  95%|▉| 322/338 [00:00<00:00, 4793.17it/s, Materializing param=model.layers.26.self_attn.q_proj.bias]
Loading weights:  95%|▉| 322/338 [00:00<00:00, 4789.63it/s, Materializing param=model.layers.26.self_attn.q_proj.bias]
Loading weights:  96%|▉| 323/338 [00:00<00:00, 4799.40it/s, Materializing param=model.layers.26.self_attn.q_proj.weigh
Loading weights:  96%|▉| 323/338 [00:00<00:00, 4795.90it/s, Materializing param=model.layers.26.self_attn.q_proj.weigh
Loading weights:  96%|▉| 324/338 [00:00<00:00, 4805.46it/s, Materializing param=model.layers.26.self_attn.v_proj.bias]
Loading weights:  96%|▉| 324/338 [00:00<00:00, 4801.96it/s, Materializing param=model.layers.26.self_attn.v_proj.bias]
Loading weights:  96%|▉| 325/338 [00:00<00:00, 4811.56it/s, Materializing param=model.layers.26.self_attn.v_proj.weigh
Loading weights:  96%|▉| 325/338 [00:00<00:00, 4808.13it/s, Materializing param=model.layers.26.self_attn.v_proj.weigh
Loading weights:  96%|▉| 326/338 [00:00<00:00, 4817.66it/s, Materializing param=model.layers.27.input_layernorm.weight
Loading weights:  96%|▉| 326/338 [00:00<00:00, 4814.18it/s, Materializing param=model.layers.27.input_layernorm.weight
Loading weights:  97%|█▉| 327/338 [00:00<00:00, 4823.72it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  97%|█▉| 327/338 [00:00<00:00, 4820.21it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  97%|█▉| 328/338 [00:00<00:00, 4829.69it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  97%|█▉| 328/338 [00:00<00:00, 4826.18it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  97%|███▉| 329/338 [00:00<00:00, 4835.69it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  97%|███▉| 329/338 [00:00<00:00, 4832.27it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  98%|▉| 330/338 [00:00<00:00, 4841.85it/s, Materializing param=model.layers.27.post_attention_layerno
Loading weights:  98%|▉| 330/338 [00:00<00:00, 4838.38it/s, Materializing param=model.layers.27.post_attention_layerno
Loading weights:  98%|▉| 331/338 [00:00<00:00, 4847.84it/s, Materializing param=model.layers.27.self_attn.k_proj.bias]
Loading weights:  98%|▉| 331/338 [00:00<00:00, 4844.42it/s, Materializing param=model.layers.27.self_attn.k_proj.bias]
Loading weights:  98%|▉| 332/338 [00:00<00:00, 4853.81it/s, Materializing param=model.layers.27.self_attn.k_proj.weigh
Loading weights:  98%|▉| 332/338 [00:00<00:00, 4850.41it/s, Materializing param=model.layers.27.self_attn.k_proj.weigh
Loading weights:  99%|▉| 333/338 [00:00<00:00, 4859.69it/s, Materializing param=model.layers.27.self_attn.o_proj.weigh
Loading weights:  99%|▉| 333/338 [00:00<00:00, 4856.12it/s, Materializing param=model.layers.27.self_attn.o_proj.weigh
Loading weights:  99%|▉| 334/338 [00:00<00:00, 4865.46it/s, Materializing param=model.layers.27.self_attn.q_proj.bias]
Loading weights:  99%|▉| 334/338 [00:00<00:00, 4861.83it/s, Materializing param=model.layers.27.self_attn.q_proj.bias]
Loading weights:  99%|▉| 335/338 [00:00<00:00, 4871.20it/s, Materializing param=model.layers.27.self_attn.q_proj.weigh
Loading weights:  99%|▉| 335/338 [00:00<00:00, 4867.75it/s, Materializing param=model.layers.27.self_attn.q_proj.weigh
Loading weights:  99%|▉| 336/338 [00:00<00:00, 4877.11it/s, Materializing param=model.layers.27.self_attn.v_proj.bias]
Loading weights:  99%|▉| 336/338 [00:00<00:00, 4873.71it/s, Materializing param=model.layers.27.self_attn.v_proj.bias]
Loading weights: 100%|▉| 337/338 [00:00<00:00, 4882.81it/s, Materializing param=model.layers.27.self_attn.v_proj.weigh
Loading weights: 100%|▉| 337/338 [00:00<00:00, 4879.30it/s, Materializing param=model.layers.27.self_attn.v_proj.weigh
Loading weights: 100%|█████████████████████| 338/338 [00:00<00:00, 4888.60it/s, Materializing param=model.norm.weight]
Loading weights: 100%|█████████████████████| 338/338 [00:00<00:00, 4885.11it/s, Materializing param=model.norm.weight]
Loading weights: 100%|█████████████████████| 338/338 [00:00<00:00, 4877.85it/s, Materializing param=model.norm.weight]
+[2026-02-10 13:03:24,490] [WARNING] [torchao.<module>:39] [PID:6544] Skipping import of cpp extensions due to incompatible torch version 2.9.1+cu128 for torchao version 0.13.0
+[2026-02-10 13:03:33,964] [WARNING] [accelerate.utils.dataclasses.__post_init__:1962] [PID:6544] sharding_strategy is deprecated in favor of reshard_after_forward. This will be removed in a future version of Accelerate.
+[2026-02-10 13:41:29,309] [WARNING] [py.warnings._showwarnmsg:110] [PID:6544] /root/miniconda3/envs/py3.11/lib/python3.11/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html .
   warnings.warn(