[2026-02-06 15:45:47,242] [WARNING] [huggingface_hub.utils._http._warn_on_warning_headers:779] [PID:7207] Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
[2026-02-06 15:45:48,737] [WARNING] [py.warnings._showwarnmsg:110] [PID:7207] /root/axolotl/.venv/lib/python3.11/site-packages/torch/distributed/distributed_c10d.py:4807: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
  warnings.warn(  # warn only once


Downloading (incomplete total...): 0.00B [00:00, ?B/s]

Fetching 5 files:   0%|                                                                                                                                   | 0/5 [00:00<?, ?it/s]

Fetching 5 files:  20%|████████████████████████▌                                                                                                  | 1/5 [00:30<02:01, 30.46s/it]
Fetching 5 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:30<00:00,  6.09s/it]

Download complete: : 0.00B [00:30, ?B/s]              
Download complete: : 0.00B [00:30, ?B/s]

Loading weights:   0%|                                                                                                                                  | 0/363 [00:00<?, ?it/s]
Loading weights:   0%|▏                                                                                   | 1/363 [00:00<00:00, 9731.56it/s, Materializing param=lm_head.weight]
Loading weights:   0%|▏                                                                                   | 1/363 [00:00<00:00, 4219.62it/s, Materializing param=lm_head.weight]
Loading weights:   1%|▍                                                                        | 2/363 [00:00<00:00, 4454.92it/s, Materializing param=model.embed_tokens.weight]
Loading weights:   1%|▍                                                                        | 2/363 [00:00<00:00, 3539.50it/s, Materializing param=model.embed_tokens.weight]
Loading weights:   1%|▌                                                            | 3/363 [00:00<00:00, 4051.16it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|▌                                                            | 3/363 [00:00<00:00, 3553.49it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|▋                                                              | 4/363 [00:00<00:00, 3981.30it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   1%|▋                                                              | 4/363 [00:00<00:00, 3622.02it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   1%|▊                                                              | 5/363 [00:00<00:00, 3811.62it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|▊                                                              | 5/363 [00:00<00:00, 3549.68it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   2%|█                                                                | 6/363 [00:00<00:00, 3430.92it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   2%|█                                                                | 6/363 [00:00<00:00, 3243.44it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   2%|█                                                   | 7/363 [00:00<00:00, 3500.67it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|█                                                   | 7/363 [00:00<00:00, 3339.03it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|█▎                                                          | 8/363 [00:00<00:00, 2977.06it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|█▎                                                          | 8/363 [00:00<00:00, 2752.39it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|█▍                                                          | 9/363 [00:00<00:00, 2928.30it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|█▍                                                          | 9/363 [00:00<00:00, 2749.96it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   3%|█▋                                                         | 10/363 [00:00<00:00, 2913.72it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|█▋                                                         | 10/363 [00:00<00:00, 2780.45it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|█▊                                                         | 11/363 [00:00<00:00, 2935.32it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   3%|█▊                                                         | 11/363 [00:00<00:00, 2861.94it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   3%|█▉                                                          | 12/363 [00:00<00:00, 3007.75it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   3%|█▉                                                          | 12/363 [00:00<00:00, 2937.87it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   4%|██▏                                                           | 13/363 [00:00<00:00, 3076.56it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   4%|██▏                                                           | 13/363 [00:00<00:00, 3009.66it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   4%|██▍                                                           | 14/363 [00:00<00:00, 3023.39it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|██▍                                                           | 14/363 [00:00<00:00, 2959.24it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|██▋                                                             | 15/363 [00:00<00:00, 2960.55it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   4%|██▋                                                             | 15/363 [00:00<00:00, 2870.58it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   4%|██▏                                                | 16/363 [00:00<00:00, 2975.74it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   4%|██▏                                                | 16/363 [00:00<00:00, 2897.12it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   5%|██▊                                                        | 17/363 [00:00<00:00, 2998.33it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   5%|██▊                                                        | 17/363 [00:00<00:00, 2949.21it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   5%|██▉                                                        | 18/363 [00:00<00:00, 3045.97it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|██▉                                                        | 18/363 [00:00<00:00, 2845.20it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|███                                                        | 19/363 [00:00<00:00, 2905.81it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|███                                                        | 19/363 [00:00<00:00, 2794.34it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   6%|███▎                                                       | 20/363 [00:00<00:00, 2876.16it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   6%|███▎                                                       | 20/363 [00:00<00:00, 2836.86it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   6%|███▍                                                        | 21/363 [00:00<00:00, 2917.63it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   6%|███▍                                                        | 21/363 [00:00<00:00, 2878.82it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   6%|███▊                                                          | 22/363 [00:00<00:00, 2896.62it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   6%|███▊                                                          | 22/363 [00:00<00:00, 2859.55it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   6%|███▉                                                          | 23/363 [00:00<00:00, 2921.62it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|███▉                                                          | 23/363 [00:00<00:00, 2865.30it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   7%|████▏                                                           | 24/363 [00:00<00:00, 2937.44it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   7%|████▏                                                           | 24/363 [00:00<00:00, 2886.90it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   7%|███▌                                               | 25/363 [00:00<00:00, 2954.57it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|███▌                                               | 25/363 [00:00<00:00, 2919.52it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|████▏                                                      | 26/363 [00:00<00:00, 2964.01it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|████▏                                                      | 26/363 [00:00<00:00, 2931.35it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|████▍                                                      | 27/363 [00:00<00:00, 2994.98it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|████▍                                                      | 27/363 [00:00<00:00, 2964.02it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   8%|████▌                                                      | 28/363 [00:00<00:00, 2860.15it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|████▌                                                      | 28/363 [00:00<00:00, 2830.92it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|████▋                                                      | 29/363 [00:00<00:00, 2890.70it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|████▋                                                      | 29/363 [00:00<00:00, 2808.40it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|████▉                                                       | 30/363 [00:00<00:00, 2840.00it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:   8%|████▉                                                       | 30/363 [00:00<00:00, 2755.42it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:   9%|█████▎                                                        | 31/363 [00:00<00:00, 2809.01it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   9%|█████▎                                                        | 31/363 [00:00<00:00, 2784.88it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   9%|█████▍                                                        | 32/363 [00:00<00:00, 2839.32it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|█████▍                                                        | 32/363 [00:00<00:00, 2815.91it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|█████▊                                                          | 33/363 [00:00<00:00, 2865.79it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|█████▊                                                          | 33/363 [00:00<00:00, 2842.78it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|████▊                                              | 34/363 [00:00<00:00, 2894.09it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:   9%|████▊                                              | 34/363 [00:00<00:00, 2871.19it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:  10%|█████▋                                                     | 35/363 [00:00<00:00, 2921.12it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  10%|█████▋                                                     | 35/363 [00:00<00:00, 2897.93it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  10%|█████▊                                                     | 36/363 [00:00<00:00, 2946.70it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|█████▊                                                     | 36/363 [00:00<00:00, 2924.67it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|██████                                                     | 37/363 [00:00<00:00, 2970.36it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|██████                                                     | 37/363 [00:00<00:00, 2947.17it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|██████▏                                                    | 38/363 [00:00<00:00, 2989.92it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  10%|██████▏                                                    | 38/363 [00:00<00:00, 2967.87it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  11%|██████▍                                                     | 39/363 [00:00<00:00, 3012.59it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  11%|██████▍                                                     | 39/363 [00:00<00:00, 2990.89it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  11%|██████▊                                                       | 40/363 [00:00<00:00, 3035.39it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  11%|██████▊                                                       | 40/363 [00:00<00:00, 3014.23it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  11%|███████                                                       | 41/363 [00:00<00:00, 2935.43it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|███████                                                       | 41/363 [00:00<00:00, 2913.90it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  12%|███████▍                                                        | 42/363 [00:00<00:00, 2955.02it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  12%|███████▍                                                        | 42/363 [00:00<00:00, 2928.84it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  12%|██████                                             | 43/363 [00:00<00:00, 2954.70it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  12%|██████                                             | 43/363 [00:00<00:00, 2934.85it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  12%|███████▏                                                   | 44/363 [00:00<00:00, 2973.73it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|███████▏                                                   | 44/363 [00:00<00:00, 2949.30it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|███████▎                                                   | 45/363 [00:00<00:00, 2977.64it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|███████▎                                                   | 45/363 [00:00<00:00, 2959.34it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  13%|███████▍                                                   | 46/363 [00:00<00:00, 2997.89it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|███████▍                                                   | 46/363 [00:00<00:00, 2969.46it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|███████▋                                                   | 47/363 [00:00<00:00, 2979.04it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  13%|███████▋                                                   | 47/363 [00:00<00:00, 2960.97it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  13%|███████▉                                                    | 48/363 [00:00<00:00, 2979.17it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  13%|███████▉                                                    | 48/363 [00:00<00:00, 2961.56it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  13%|████████▎                                                     | 49/363 [00:00<00:00, 2996.67it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  13%|████████▎                                                     | 49/363 [00:00<00:00, 2977.92it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  14%|████████▌                                                     | 50/363 [00:00<00:00, 2922.25it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|████████▌                                                     | 50/363 [00:00<00:00, 2905.49it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|████████▉                                                       | 51/363 [00:00<00:00, 2902.91it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|████████▉                                                       | 51/363 [00:00<00:00, 2877.87it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|███████▎                                           | 52/363 [00:00<00:00, 2909.14it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  14%|███████▎                                           | 52/363 [00:00<00:00, 2893.47it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  15%|████████▌                                                  | 53/363 [00:00<00:00, 2925.98it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  15%|████████▌                                                  | 53/363 [00:00<00:00, 2910.96it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  15%|████████▊                                                  | 54/363 [00:00<00:00, 2942.99it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|████████▊                                                  | 54/363 [00:00<00:00, 2926.52it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|████████▉                                                  | 55/363 [00:00<00:00, 2957.45it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|████████▉                                                  | 55/363 [00:00<00:00, 2942.43it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█████████                                                  | 56/363 [00:00<00:00, 2973.63it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  15%|█████████                                                  | 56/363 [00:00<00:00, 2958.72it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  16%|█████████▍                                                  | 57/363 [00:00<00:00, 2988.52it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  16%|█████████▍                                                  | 57/363 [00:00<00:00, 2973.72it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  16%|█████████▉                                                    | 58/363 [00:00<00:00, 3003.03it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  16%|█████████▉                                                    | 58/363 [00:00<00:00, 2988.35it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  16%|██████████                                                    | 59/363 [00:00<00:00, 2920.45it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|██████████                                                    | 59/363 [00:00<00:00, 2906.01it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  17%|██████████▌                                                     | 60/363 [00:00<00:00, 2933.15it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  17%|██████████▌                                                     | 60/363 [00:00<00:00, 2919.10it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  17%|████████▌                                          | 61/363 [00:00<00:00, 2946.32it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  17%|████████▌                                          | 61/363 [00:00<00:00, 2930.76it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  17%|██████████                                                 | 62/363 [00:00<00:00, 2775.91it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|██████████                                                 | 62/363 [00:00<00:00, 2721.01it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|██████████▏                                                | 63/363 [00:00<00:00, 2744.28it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|██████████▏                                                | 63/363 [00:00<00:00, 2731.40it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  18%|██████████▍                                                | 64/363 [00:00<00:00, 2756.04it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  18%|██████████▍                                                | 64/363 [00:00<00:00, 2699.85it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  18%|██████████▌                                                | 65/363 [00:00<00:00, 2723.03it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|██████████▌                                                | 65/363 [00:00<00:00, 2710.85it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|██████████▉                                                 | 66/363 [00:00<00:00, 2734.50it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  18%|██████████▉                                                 | 66/363 [00:00<00:00, 2722.61it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  18%|███████████▍                                                  | 67/363 [00:00<00:00, 2745.98it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  18%|███████████▍                                                  | 67/363 [00:00<00:00, 2733.35it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  19%|███████████▌                                                  | 68/363 [00:00<00:00, 2756.18it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|███████████▌                                                  | 68/363 [00:00<00:00, 2744.38it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|████████████▏                                                   | 69/363 [00:00<00:00, 2767.09it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|████████████▏                                                   | 69/363 [00:00<00:00, 2755.52it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|█████████▊                                         | 70/363 [00:00<00:00, 2777.90it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  19%|█████████▊                                         | 70/363 [00:00<00:00, 2765.96it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  20%|███████████▌                                               | 71/363 [00:00<00:00, 2787.33it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  20%|███████████▌                                               | 71/363 [00:00<00:00, 2775.38it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  20%|███████████▋                                               | 72/363 [00:00<00:00, 2695.26it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  20%|███████████▋                                               | 72/363 [00:00<00:00, 2679.16it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  20%|███████████▊                                               | 73/363 [00:00<00:00, 2698.78it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|███████████▊                                               | 73/363 [00:00<00:00, 2688.14it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|████████████                                               | 74/363 [00:00<00:00, 2709.10it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  20%|████████████                                               | 74/363 [00:00<00:00, 2698.71it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  21%|████████████▍                                               | 75/363 [00:00<00:00, 2718.54it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  21%|████████████▍                                               | 75/363 [00:00<00:00, 2708.22it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  21%|████████████▉                                                 | 76/363 [00:00<00:00, 2729.33it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  21%|████████████▉                                                 | 76/363 [00:00<00:00, 2718.77it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  21%|█████████████▏                                                | 77/363 [00:00<00:00, 2703.67it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|█████████████▏                                                | 77/363 [00:00<00:00, 2692.71it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|█████████████▊                                                  | 78/363 [00:00<00:00, 2712.67it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  21%|█████████████▊                                                  | 78/363 [00:00<00:00, 2702.69it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  22%|███████████                                        | 79/363 [00:00<00:00, 2722.41it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  22%|███████████                                        | 79/363 [00:00<00:00, 2712.18it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  22%|█████████████                                              | 80/363 [00:00<00:00, 2731.62it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|█████████████                                              | 80/363 [00:00<00:00, 2721.43it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|█████████████▏                                             | 81/363 [00:00<00:00, 2701.66it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|█████████████▏                                             | 81/363 [00:00<00:00, 2691.34it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  23%|█████████████▎                                             | 82/363 [00:00<00:00, 2688.38it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|█████████████▎                                             | 82/363 [00:00<00:00, 2622.08it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|█████████████▍                                             | 83/363 [00:00<00:00, 2637.55it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|█████████████▍                                             | 83/363 [00:00<00:00, 2628.11it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|█████████████▉                                              | 84/363 [00:00<00:00, 2646.07it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  23%|█████████████▉                                              | 84/363 [00:00<00:00, 2637.22it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  23%|██████████████▌                                               | 85/363 [00:00<00:00, 2655.37it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  23%|██████████████▌                                               | 85/363 [00:00<00:00, 2646.58it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  24%|██████████████▋                                               | 86/363 [00:00<00:00, 2664.70it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  24%|██████████████▋                                               | 86/363 [00:00<00:00, 2656.09it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  24%|███████████████▎                                                | 87/363 [00:00<00:00, 2655.47it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  24%|███████████████▎                                                | 87/363 [00:00<00:00, 2646.52it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  24%|████████████▎                                      | 88/363 [00:00<00:00, 2579.49it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  24%|████████████▎                                      | 88/363 [00:00<00:00, 2570.92it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  25%|██████████████▍                                            | 89/363 [00:00<00:00, 2588.09it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  25%|██████████████▍                                            | 89/363 [00:00<00:00, 2579.49it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  25%|██████████████▋                                            | 90/363 [00:00<00:00, 2596.31it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██████████████▋                                            | 90/363 [00:00<00:00, 2588.10it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██████████████▊                                            | 91/363 [00:00<00:00, 2577.05it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██████████████▊                                            | 91/363 [00:00<00:00, 2568.29it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██████████████▉                                            | 92/363 [00:00<00:00, 2584.34it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  25%|██████████████▉                                            | 92/363 [00:00<00:00, 2571.51it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  26%|███████████████                                            | 93/363 [00:00<00:00, 2586.90it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|███████████████                                            | 93/363 [00:00<00:00, 2578.82it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|███████████████▊                                             | 94/363 [00:00<00:00, 2593.81it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  26%|███████████████▊                                             | 94/363 [00:00<00:00, 2585.95it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  26%|███████████████▉                                             | 95/363 [00:00<00:00, 2545.66it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|███████████████▉                                             | 95/363 [00:00<00:00, 2536.06it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|████████████████▋                                              | 96/363 [00:00<00:00, 2550.63it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  26%|████████████████▋                                              | 96/363 [00:00<00:00, 2542.39it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  27%|█████████████▎                                    | 97/363 [00:00<00:00, 2554.84it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  27%|█████████████▎                                    | 97/363 [00:00<00:00, 2541.67it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  27%|███████████████▋                                          | 98/363 [00:00<00:00, 2551.82it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|███████████████▋                                          | 98/363 [00:00<00:00, 2542.51it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|███████████████▊                                          | 99/363 [00:00<00:00, 2518.49it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|███████████████▊                                          | 99/363 [00:00<00:00, 2509.25it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  28%|███████████████▋                                         | 100/363 [00:00<00:00, 2523.04it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|███████████████▋                                         | 100/363 [00:00<00:00, 2515.63it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|███████████████▊                                         | 101/363 [00:00<00:00, 2529.34it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|███████████████▊                                         | 101/363 [00:00<00:00, 2522.19it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|████████████████▎                                         | 102/363 [00:00<00:00, 2515.13it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights:  28%|████████████████▎                                         | 102/363 [00:00<00:00, 2488.43it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights:  28%|█████████████████                                           | 103/363 [00:00<00:00, 2501.84it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  28%|█████████████████                                           | 103/363 [00:00<00:00, 2495.10it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  29%|█████████████████▏                                          | 104/363 [00:00<00:00, 2507.81it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  29%|█████████████████▏                                          | 104/363 [00:00<00:00, 2500.87it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  29%|█████████████████▉                                            | 105/363 [00:00<00:00, 2514.04it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  29%|█████████████████▉                                            | 105/363 [00:00<00:00, 2506.70it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  29%|██████████████▎                                  | 106/363 [00:00<00:00, 2475.76it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██████████████▎                                  | 106/363 [00:00<00:00, 2466.79it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|████████████████▊                                        | 107/363 [00:00<00:00, 2478.86it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  29%|████████████████▊                                        | 107/363 [00:00<00:00, 2471.87it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  30%|████████████████▉                                        | 108/363 [00:00<00:00, 2483.29it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|████████████████▉                                        | 108/363 [00:00<00:00, 2476.50it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|█████████████████                                        | 109/363 [00:00<00:00, 2451.65it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|█████████████████                                        | 109/363 [00:00<00:00, 2445.24it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|█████████████████▎                                       | 110/363 [00:00<00:00, 2457.46it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|█████████████████▎                                       | 110/363 [00:00<00:00, 2451.12it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  31%|█████████████████▋                                        | 111/363 [00:00<00:00, 2463.63it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights:  31%|█████████████████▋                                        | 111/363 [00:00<00:00, 2457.12it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights:  31%|██████████████████▌                                         | 112/363 [00:00<00:00, 2432.95it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  31%|██████████████████▌                                         | 112/363 [00:00<00:00, 2423.23it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  31%|██████████████████▋                                         | 113/363 [00:00<00:00, 2435.17it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|██████████████████▋                                         | 113/363 [00:00<00:00, 2427.83it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███████████████████▍                                          | 114/363 [00:00<00:00, 2440.27it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  31%|███████████████████▍                                          | 114/363 [00:00<00:00, 2434.06it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  32%|███████████████▌                                 | 115/363 [00:00<00:00, 2421.75it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  32%|███████████████▌                                 | 115/363 [00:00<00:00, 2415.15it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  32%|██████████████████▏                                      | 116/363 [00:00<00:00, 2426.94it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  32%|██████████████████▏                                      | 116/363 [00:00<00:00, 2421.03it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  32%|██████████████████▎                                      | 117/363 [00:00<00:00, 2431.25it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|██████████████████▎                                      | 117/363 [00:00<00:00, 2422.81it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  33%|██████████████████▌                                      | 118/363 [00:00<00:00, 2355.76it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|██████████████████▌                                      | 118/363 [00:00<00:00, 2333.57it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|██████████████████▋                                      | 119/363 [00:00<00:00, 2339.09it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  33%|██████████████████▋                                      | 119/363 [00:00<00:00, 2333.60it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  33%|███████████████████▏                                      | 120/363 [00:00<00:00, 2344.31it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights:  33%|███████████████████▏                                      | 120/363 [00:00<00:00, 2330.05it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights:  33%|████████████████████                                        | 121/363 [00:00<00:00, 2338.87it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  33%|████████████████████                                        | 121/363 [00:00<00:00, 2313.48it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  34%|████████████████████▏                                       | 122/363 [00:00<00:00, 2314.87it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  34%|████████████████████▏                                       | 122/363 [00:00<00:00, 2309.14it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  34%|█████████████████████                                         | 123/363 [00:00<00:00, 2319.62it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  34%|█████████████████████                                         | 123/363 [00:00<00:00, 2314.03it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  34%|████████████████▋                                | 124/363 [00:00<00:00, 2324.95it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|████████████████▋                                | 124/363 [00:00<00:00, 2318.73it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|███████████████████▋                                     | 125/363 [00:00<00:00, 2329.12it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  34%|███████████████████▋                                     | 125/363 [00:00<00:00, 2323.99it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  35%|███████████████████▊                                     | 126/363 [00:00<00:00, 2334.45it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███████████████████▊                                     | 126/363 [00:00<00:00, 2329.33it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███████████████████▉                                     | 127/363 [00:00<00:00, 2339.53it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███████████████████▉                                     | 127/363 [00:00<00:00, 2334.34it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|████████████████████                                     | 128/363 [00:00<00:00, 2344.17it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  35%|████████████████████                                     | 128/363 [00:00<00:00, 2338.79it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  36%|████████████████████▌                                     | 129/363 [00:00<00:00, 2348.81it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights:  36%|████████████████████▌                                     | 129/363 [00:00<00:00, 2343.72it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights:  36%|█████████████████████▍                                      | 130/363 [00:00<00:00, 2354.32it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  36%|█████████████████████▍                                      | 130/363 [00:00<00:00, 2348.72it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  36%|█████████████████████▋                                      | 131/363 [00:00<00:00, 2358.11it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|█████████████████████▋                                      | 131/363 [00:00<00:00, 2352.82it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|██████████████████████▌                                       | 132/363 [00:00<00:00, 2362.65it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  36%|██████████████████████▌                                       | 132/363 [00:00<00:00, 2355.59it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  37%|█████████████████▉                               | 133/363 [00:00<00:00, 2352.63it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  37%|█████████████████▉                               | 133/363 [00:00<00:00, 2344.30it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  37%|█████████████████████                                    | 134/363 [00:00<00:00, 2339.77it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|█████████████████████                                    | 134/363 [00:00<00:00, 2334.45it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|█████████████████████▏                                   | 135/363 [00:00<00:00, 2337.36it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|█████████████████████▏                                   | 135/363 [00:00<00:00, 2330.02it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|█████████████████████▎                                   | 136/363 [00:00<00:00, 2338.97it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  37%|█████████████████████▎                                   | 136/363 [00:00<00:00, 2332.58it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  38%|█████████████████████▌                                   | 137/363 [00:00<00:00, 2325.07it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  38%|█████████████████████▌                                   | 137/363 [00:00<00:00, 2319.43it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  38%|██████████████████████                                    | 138/363 [00:00<00:00, 2329.11it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights:  38%|██████████████████████                                    | 138/363 [00:00<00:00, 2324.36it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights:  38%|██████████████████████▉                                     | 139/363 [00:00<00:00, 2326.19it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  38%|██████████████████████▉                                     | 139/363 [00:00<00:00, 2321.30it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  39%|███████████████████████▏                                    | 140/363 [00:00<00:00, 2328.88it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  39%|███████████████████████▏                                    | 140/363 [00:00<00:00, 2317.12it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  39%|████████████████████████                                      | 141/363 [00:00<00:00, 2307.38it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|████████████████████████                                      | 141/363 [00:00<00:00, 2302.64it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|███████████████████▏                             | 142/363 [00:00<00:00, 2311.88it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|███████████████████▏                             | 142/363 [00:00<00:00, 2306.45it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|██████████████████████▍                                  | 143/363 [00:00<00:00, 2304.73it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  39%|██████████████████████▍                                  | 143/363 [00:00<00:00, 2299.59it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  40%|██████████████████████▌                                  | 144/363 [00:00<00:00, 2301.91it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|██████████████████████▌                                  | 144/363 [00:00<00:00, 2296.72it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|██████████████████████▊                                  | 145/363 [00:00<00:00, 2304.21it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|██████████████████████▊                                  | 145/363 [00:00<00:00, 2295.23it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|██████████████████████▉                                  | 146/363 [00:00<00:00, 2300.95it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|██████████████████████▉                                  | 146/363 [00:00<00:00, 2293.88it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|███████████████████████▍                                  | 147/363 [00:00<00:00, 2283.40it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  40%|███████████████████████▍                                  | 147/363 [00:00<00:00, 2269.04it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  41%|████████████████████████▍                                   | 148/363 [00:00<00:00, 2277.69it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  41%|████████████████████████▍                                   | 148/363 [00:00<00:00, 2265.02it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  41%|████████████████████████▋                                   | 149/363 [00:00<00:00, 2270.49it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████████████████████████▋                                   | 149/363 [00:00<00:00, 2260.58it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|█████████████████████████▌                                    | 150/363 [00:00<00:00, 2269.25it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  41%|█████████████████████████▌                                    | 150/363 [00:00<00:00, 2264.13it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  42%|████████████████████▍                            | 151/363 [00:00<00:00, 2264.13it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  42%|████████████████████▍                            | 151/363 [00:00<00:00, 2259.97it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  42%|███████████████████████▊                                 | 152/363 [00:00<00:00, 2268.66it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  42%|███████████████████████▊                                 | 152/363 [00:00<00:00, 2262.84it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  42%|████████████████████████                                 | 153/363 [00:00<00:00, 2271.57it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████████████████████████                                 | 153/363 [00:00<00:00, 2265.24it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████████████████████████▏                                | 154/363 [00:00<00:00, 2273.39it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  42%|████████████████████████▏                                | 154/363 [00:00<00:00, 2255.96it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  43%|████████████████████████▎                                | 155/363 [00:00<00:00, 2254.12it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  43%|████████████████████████▎                                | 155/363 [00:00<00:00, 2246.72it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  43%|████████████████████████▉                                 | 156/363 [00:00<00:00, 2244.22it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights:  43%|████████████████████████▉                                 | 156/363 [00:00<00:00, 2238.32it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights:  43%|█████████████████████████▉                                  | 157/363 [00:00<00:00, 2246.64it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  43%|█████████████████████████▉                                  | 157/363 [00:00<00:00, 2236.27it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  44%|██████████████████████████                                  | 158/363 [00:00<00:00, 2244.55it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  44%|██████████████████████████                                  | 158/363 [00:00<00:00, 2233.64it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  44%|███████████████████████████▏                                  | 159/363 [00:00<00:00, 2241.38it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|███████████████████████████▏                                  | 159/363 [00:00<00:00, 2237.30it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|█████████████████████▌                           | 160/363 [00:00<00:00, 2245.65it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|█████████████████████▌                           | 160/363 [00:00<00:00, 2233.31it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|█████████████████████████▎                               | 161/363 [00:00<00:00, 2238.82it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|█████████████████████████▎                               | 161/363 [00:00<00:00, 2231.98it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  45%|█████████████████████████▍                               | 162/363 [00:00<00:00, 2237.30it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  45%|█████████████████████████▍                               | 162/363 [00:00<00:00, 2224.11it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  45%|█████████████████████████▌                               | 163/363 [00:00<00:00, 2230.71it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|█████████████████████████▌                               | 163/363 [00:00<00:00, 2224.93it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|█████████████████████████▊                               | 164/363 [00:00<00:00, 2230.51it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|█████████████████████████▊                               | 164/363 [00:00<00:00, 2226.78it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|██████████████████████████▎                               | 165/363 [00:00<00:00, 2235.00it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights:  45%|██████████████████████████▎                               | 165/363 [00:00<00:00, 2229.87it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights:  46%|███████████████████████████▍                                | 166/363 [00:00<00:00, 2238.00it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  46%|███████████████████████████▍                                | 166/363 [00:00<00:00, 2234.43it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  46%|███████████████████████████▌                                | 167/363 [00:00<00:00, 2238.58it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|███████████████████████████▌                                | 167/363 [00:00<00:00, 2234.75it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████████████████████████████▋                                 | 168/363 [00:00<00:00, 2242.74it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  46%|████████████████████████████▋                                 | 168/363 [00:00<00:00, 2239.14it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  47%|██████████████████████▊                          | 169/363 [00:00<00:00, 2247.08it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  47%|██████████████████████▊                          | 169/363 [00:00<00:00, 2243.43it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  47%|██████████████████████████▋                              | 170/363 [00:00<00:00, 2250.82it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|██████████████████████████▋                              | 170/363 [00:00<00:00, 2247.24it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|██████████████████████████▊                              | 171/363 [00:00<00:00, 2255.19it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|██████████████████████████▊                              | 171/363 [00:00<00:00, 2251.16it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|███████████████████████████                              | 172/363 [00:00<00:00, 2216.67it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  47%|███████████████████████████                              | 172/363 [00:00<00:00, 2213.01it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  48%|███████████████████████████▏                             | 173/363 [00:00<00:00, 2220.91it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|███████████████████████████▏                             | 173/363 [00:00<00:00, 2217.66it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|███████████████████████████▊                              | 174/363 [00:00<00:00, 2225.64it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights:  48%|███████████████████████████▊                              | 174/363 [00:00<00:00, 2222.42it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights:  48%|████████████████████████████▉                               | 175/363 [00:00<00:00, 2230.10it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  48%|████████████████████████████▉                               | 175/363 [00:00<00:00, 2226.74it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  48%|█████████████████████████████                               | 176/363 [00:00<00:00, 2234.29it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  48%|█████████████████████████████                               | 176/363 [00:00<00:00, 2231.05it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  49%|██████████████████████████████▏                               | 177/363 [00:00<00:00, 2238.95it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  49%|██████████████████████████████▏                               | 177/363 [00:00<00:00, 2235.78it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  49%|████████████████████████                         | 178/363 [00:00<00:00, 2243.61it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████████████████████████                         | 178/363 [00:00<00:00, 2240.02it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████████████████████████████                             | 179/363 [00:00<00:00, 2247.54it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████████████████████████████                             | 179/363 [00:00<00:00, 2244.29it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  50%|████████████████████████████▎                            | 180/363 [00:00<00:00, 2252.08it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  50%|████████████████████████████▎                            | 180/363 [00:00<00:00, 2248.84it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  50%|████████████████████████████▍                            | 181/363 [00:00<00:00, 2256.54it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|████████████████████████████▍                            | 181/363 [00:00<00:00, 2253.32it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|████████████████████████████▌                            | 182/363 [00:00<00:00, 2246.59it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|████████████████████████████▌                            | 182/363 [00:00<00:00, 2243.17it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████████████████████████████▏                            | 183/363 [00:00<00:00, 2250.72it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights:  50%|█████████████████████████████▏                            | 183/363 [00:00<00:00, 2247.18it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights:  51%|██████████████████████████████▍                             | 184/363 [00:00<00:00, 2254.51it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  51%|██████████████████████████████▍                             | 184/363 [00:00<00:00, 2251.27it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  51%|██████████████████████████████▌                             | 185/363 [00:00<00:00, 2254.89it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|██████████████████████████████▌                             | 185/363 [00:00<00:00, 2251.52it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|███████████████████████████████▊                              | 186/363 [00:00<00:00, 2246.86it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  51%|███████████████████████████████▊                              | 186/363 [00:00<00:00, 2238.10it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  52%|█████████████████████████▏                       | 187/363 [00:00<00:00, 2245.24it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  52%|█████████████████████████▏                       | 187/363 [00:00<00:00, 2242.05it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  52%|█████████████████████████████▌                           | 188/363 [00:00<00:00, 2248.91it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████████████████████████████▌                           | 188/363 [00:00<00:00, 2245.72it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████████████████████████████▋                           | 189/363 [00:00<00:00, 2252.90it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████████████████████████████▋                           | 189/363 [00:00<00:00, 2249.74it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████████████████████████████▊                           | 190/363 [00:00<00:00, 2244.62it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  52%|█████████████████████████████▊                           | 190/363 [00:00<00:00, 2241.26it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  53%|█████████████████████████████▉                           | 191/363 [00:00<00:00, 2248.20it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  53%|█████████████████████████████▉                           | 191/363 [00:00<00:00, 2245.02it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  53%|██████████████████████████████▋                           | 192/363 [00:00<00:00, 2238.95it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights:  53%|██████████████████████████████▋                           | 192/363 [00:00<00:00, 2235.30it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights:  53%|███████████████████████████████▉                            | 193/363 [00:00<00:00, 2242.12it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  53%|███████████████████████████████▉                            | 193/363 [00:00<00:00, 2238.87it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  53%|████████████████████████████████                            | 194/363 [00:00<00:00, 2228.58it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|████████████████████████████████                            | 194/363 [00:00<00:00, 2225.27it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  54%|█████████████████████████████████▎                            | 195/363 [00:00<00:00, 2228.23it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  54%|█████████████████████████████████▎                            | 195/363 [00:00<00:00, 2225.13it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  54%|██████████████████████████▍                      | 196/363 [00:00<00:00, 2220.50it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|██████████████████████████▍                      | 196/363 [00:00<00:00, 2217.42it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|██████████████████████████████▉                          | 197/363 [00:00<00:00, 2223.89it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|██████████████████████████████▉                          | 197/363 [00:00<00:00, 2220.86it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  55%|███████████████████████████████                          | 198/363 [00:00<00:00, 2227.63it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  55%|███████████████████████████████                          | 198/363 [00:00<00:00, 2224.62it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  55%|███████████████████████████████▏                         | 199/363 [00:00<00:00, 2231.43it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|███████████████████████████████▏                         | 199/363 [00:00<00:00, 2228.48it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|███████████████████████████████▍                         | 200/363 [00:00<00:00, 2235.06it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|███████████████████████████████▍                         | 200/363 [00:00<00:00, 2231.96it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|████████████████████████████████                          | 201/363 [00:00<00:00, 2218.95it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights:  55%|████████████████████████████████                          | 201/363 [00:00<00:00, 2215.40it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights:  56%|█████████████████████████████████▍                          | 202/363 [00:00<00:00, 2221.83it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  56%|█████████████████████████████████▍                          | 202/363 [00:00<00:00, 2218.94it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  56%|█████████████████████████████████▌                          | 203/363 [00:00<00:00, 2210.92it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████████████████████████████████▌                          | 203/363 [00:00<00:00, 2207.87it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|██████████████████████████████████▊                           | 204/363 [00:00<00:00, 2214.38it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  56%|██████████████████████████████████▊                           | 204/363 [00:00<00:00, 2211.53it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  56%|███████████████████████████▋                     | 205/363 [00:00<00:00, 2218.11it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  56%|███████████████████████████▋                     | 205/363 [00:00<00:00, 2214.96it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  57%|████████████████████████████████▎                        | 206/363 [00:00<00:00, 2208.39it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  57%|████████████████████████████████▎                        | 206/363 [00:00<00:00, 2205.48it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  57%|████████████████████████████████▌                        | 207/363 [00:00<00:00, 2211.11it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|████████████████████████████████▌                        | 207/363 [00:00<00:00, 2208.25it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|████████████████████████████████▋                        | 208/363 [00:00<00:00, 2214.81it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  57%|████████████████████████████████▋                        | 208/363 [00:00<00:00, 2212.05it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  58%|████████████████████████████████▊                        | 209/363 [00:00<00:00, 2218.58it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|████████████████████████████████▊                        | 209/363 [00:00<00:00, 2215.80it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████████████████████████████████▌                        | 210/363 [00:00<00:00, 2217.62it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights:  58%|█████████████████████████████████▌                        | 210/363 [00:00<00:00, 2214.60it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights:  58%|██████████████████████████████████▉                         | 211/363 [00:00<00:00, 2189.96it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|██████████████████████████████████▉                         | 211/363 [00:00<00:00, 2186.74it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|███████████████████████████████████                         | 212/363 [00:00<00:00, 2192.86it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|███████████████████████████████████                         | 212/363 [00:00<00:00, 2190.11it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  59%|████████████████████████████████████▍                         | 213/363 [00:00<00:00, 2196.43it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  59%|████████████████████████████████████▍                         | 213/363 [00:00<00:00, 2193.75it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  59%|████████████████████████████▉                    | 214/363 [00:00<00:00, 2200.03it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|████████████████████████████▉                    | 214/363 [00:00<00:00, 2197.27it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|█████████████████████████████████▊                       | 215/363 [00:00<00:00, 2202.73it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████████████████████████████████▊                       | 215/363 [00:00<00:00, 2200.00it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  60%|█████████████████████████████████▉                       | 216/363 [00:00<00:00, 2206.24it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  60%|█████████████████████████████████▉                       | 216/363 [00:00<00:00, 2203.57it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  60%|██████████████████████████████████                       | 217/363 [00:00<00:00, 2209.76it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|██████████████████████████████████                       | 217/363 [00:00<00:00, 2207.05it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|██████████████████████████████████▏                      | 218/363 [00:00<00:00, 2198.17it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████████████████████████████████▏                      | 218/363 [00:00<00:00, 2195.37it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████████████████████████████████▉                       | 219/363 [00:00<00:00, 2201.17it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights:  60%|██████████████████████████████████▉                       | 219/363 [00:00<00:00, 2198.40it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights:  61%|███████████████████████████████████▏                      | 220/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights:  61%|████████████████████████████████████▎                       | 220/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  61%|████████████████████████████████████▎                       | 220/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  61%|████████████████████████████████████▌                       | 221/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|████████████████████████████████████▌                       | 221/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|█████████████████████████████████████▉                        | 222/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|█████████████████████████████████████▉                        | 222/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|██████████████████████████████                   | 223/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  61%|██████████████████████████████                   | 223/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  62%|███████████████████████████████████▏                     | 224/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  62%|███████████████████████████████████▏                     | 224/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  62%|███████████████████████████████████▎                     | 225/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|███████████████████████████████████▎                     | 225/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|███████████████████████████████████▍                     | 226/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  62%|███████████████████████████████████▍                     | 226/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  63%|███████████████████████████████████▋                     | 227/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|███████████████████████████████████▋                     | 227/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|████████████████████████████████████▍                     | 228/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights:  63%|████████████████████████████████████▍                     | 228/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights:  63%|█████████████████████████████████████▊                      | 229/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  63%|█████████████████████████████████████▊                      | 229/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  63%|██████████████████████████████████████                      | 230/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████████████████████████████████████                      | 230/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  64%|███████████████████████████████████████▍                      | 231/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  64%|███████████████████████████████████████▍                      | 231/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  64%|███████████████████████████████▎                 | 232/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  64%|███████████████████████████████▎                 | 232/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  64%|████████████████████████████████████▌                    | 233/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|████████████████████████████████████▌                    | 233/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|████████████████████████████████████▋                    | 234/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  64%|████████████████████████████████████▋                    | 234/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  65%|████████████████████████████████████▉                    | 235/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|████████████████████████████████████▉                    | 235/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|█████████████████████████████████████                    | 236/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|█████████████████████████████████████                    | 236/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|█████████████████████████████████████▊                    | 237/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights:  65%|█████████████████████████████████████▊                    | 237/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights:  66%|███████████████████████████████████████▎                    | 238/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  66%|███████████████████████████████████████▎                    | 238/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  66%|███████████████████████████████████████▌                    | 239/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|███████████████████████████████████████▌                    | 239/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|████████████████████████████████████████▉                     | 240/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  66%|████████████████████████████████████████▉                     | 240/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  66%|████████████████████████████████▌                | 241/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  66%|████████████████████████████████▌                | 241/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  67%|██████████████████████████████████████                   | 242/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  67%|██████████████████████████████████████                   | 242/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  67%|██████████████████████████████████████▏                  | 243/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████████████████████████████████████▏                  | 243/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████████████████████████████████████▎                  | 244/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████████████████████████████████████▎                  | 244/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████████████████████████████████████▍                  | 245/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  67%|██████████████████████████████████████▍                  | 245/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  68%|███████████████████████████████████████▎                  | 246/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights:  68%|███████████████████████████████████████▎                  | 246/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights:  68%|████████████████████████████████████████▊                   | 247/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  68%|████████████████████████████████████████▊                   | 247/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  68%|████████████████████████████████████████▉                   | 248/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|████████████████████████████████████████▉                   | 248/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  69%|██████████████████████████████████████████▌                   | 249/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  69%|██████████████████████████████████████████▌                   | 249/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  69%|█████████████████████████████████▋               | 250/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|█████████████████████████████████▋               | 250/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|███████████████████████████████████████▍                 | 251/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|███████████████████████████████████████▍                 | 251/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|███████████████████████████████████████▌                 | 252/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  69%|███████████████████████████████████████▌                 | 252/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  70%|███████████████████████████████████████▋                 | 253/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|███████████████████████████████████████▋                 | 253/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|███████████████████████████████████████▉                 | 254/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|███████████████████████████████████████▉                 | 254/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|████████████████████████████████████████▋                 | 255/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.input_layernorm.weight]
Loading weights:  70%|████████████████████████████████████████▋                 | 255/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.input_layernorm.weight]
Loading weights:  71%|██████████████████████████████████████████▎                 | 256/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
Loading weights:  71%|██████████████████████████████████████████▎                 | 256/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
Loading weights:  71%|██████████████████████████████████████████▍                 | 257/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|██████████████████████████████████████████▍                 | 257/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|████████████████████████████████████████████                  | 258/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|████████████████████████████████████████████                  | 258/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|██████████████████████████████████▉              | 259/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  71%|██████████████████████████████████▉              | 259/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  72%|████████████████████████████████████████▊                | 260/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  72%|████████████████████████████████████████▊                | 260/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  72%|████████████████████████████████████████▉                | 261/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|████████████████████████████████████████▉                | 261/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|█████████████████████████████████████████▏               | 262/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|█████████████████████████████████████████▏               | 262/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|█████████████████████████████████████████▎               | 263/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  72%|█████████████████████████████████████████▎               | 263/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  73%|██████████████████████████████████████████▏               | 264/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.input_layernorm.weight]
Loading weights:  73%|██████████████████████████████████████████▏               | 264/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.input_layernorm.weight]
Loading weights:  73%|███████████████████████████████████████████▊                | 265/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
Loading weights:  73%|███████████████████████████████████████████▊                | 265/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
Loading weights:  73%|███████████████████████████████████████████▉                | 266/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████████████████████████████████████████▉                | 266/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  74%|█████████████████████████████████████████████▌                | 267/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  74%|█████████████████████████████████████████████▌                | 267/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  74%|████████████████████████████████████▏            | 268/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|████████████████████████████████████▏            | 268/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|██████████████████████████████████████████▏              | 269/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|██████████████████████████████████████████▏              | 269/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|██████████████████████████████████████████▍              | 270/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|██████████████████████████████████████████▍              | 270/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  75%|██████████████████████████████████████████▌              | 271/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|██████████████████████████████████████████▌              | 271/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|██████████████████████████████████████████▋              | 272/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|██████████████████████████████████████████▋              | 272/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|███████████████████████████████████████████▌              | 273/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.input_layernorm.weight]
Loading weights:  75%|███████████████████████████████████████████▌              | 273/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.input_layernorm.weight]
Loading weights:  75%|█████████████████████████████████████████████▎              | 274/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
Loading weights:  75%|█████████████████████████████████████████████▎              | 274/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
Loading weights:  76%|█████████████████████████████████████████████▍              | 275/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|█████████████████████████████████████████████▍              | 275/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████████████████████████████████████████████▏              | 276/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|███████████████████████████████████████████████▏              | 276/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|█████████████████████████████████████▍           | 277/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  76%|█████████████████████████████████████▍           | 277/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  77%|███████████████████████████████████████████▋             | 278/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████████████████████████████████████████▋             | 278/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████████████████████████████████████████▊             | 279/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████████████████████████████████████████▊             | 279/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████████████████████████████████████████▉             | 280/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████████████████████████████████████████▉             | 280/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|████████████████████████████████████████████             | 281/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  77%|████████████████████████████████████████████             | 281/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  78%|█████████████████████████████████████████████             | 282/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.input_layernorm.weight]
Loading weights:  78%|█████████████████████████████████████████████             | 282/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.input_layernorm.weight]
Loading weights:  78%|██████████████████████████████████████████████▊             | 283/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
Loading weights:  78%|██████████████████████████████████████████████▊             | 283/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
Loading weights:  78%|██████████████████████████████████████████████▉             | 284/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|██████████████████████████████████████████████▉             | 284/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  79%|████████████████████████████████████████████████▋             | 285/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
Loading weights:  79%|████████████████████████████████████████████████▋             | 285/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
Loading weights:  79%|██████████████████████████████████████▌          | 286/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|██████████████████████████████████████▌          | 286/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|█████████████████████████████████████████████            | 287/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  79%|█████████████████████████████████████████████            | 287/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  79%|█████████████████████████████████████████████▏           | 288/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|█████████████████████████████████████████████▏           | 288/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  80%|█████████████████████████████████████████████▍           | 289/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|█████████████████████████████████████████████▍           | 289/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|█████████████████████████████████████████████▌           | 290/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  80%|█████████████████████████████████████████████▌           | 290/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  80%|██████████████████████████████████████████████▍           | 291/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.input_layernorm.weight]
Loading weights:  80%|██████████████████████████████████████████████▍           | 291/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.input_layernorm.weight]
Loading weights:  80%|████████████████████████████████████████████████▎           | 292/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
Loading weights:  80%|████████████████████████████████████████████████▎           | 292/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
Loading weights:  81%|████████████████████████████████████████████████▍           | 293/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████████████████████████████████████████████▍           | 293/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|██████████████████████████████████████████████████▏           | 294/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
Loading weights:  81%|██████████████████████████████████████████████████▏           | 294/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
Loading weights:  81%|███████████████████████████████████████▊         | 295/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  81%|███████████████████████████████████████▊         | 295/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  82%|██████████████████████████████████████████████▍          | 296/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  82%|██████████████████████████████████████████████▍          | 296/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  82%|██████████████████████████████████████████████▋          | 297/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|██████████████████████████████████████████████▋          | 297/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|██████████████████████████████████████████████▊          | 298/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|██████████████████████████████████████████████▊          | 298/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|██████████████████████████████████████████████▉          | 299/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  82%|██████████████████████████████████████████████▉          | 299/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  83%|███████████████████████████████████████████████▉          | 300/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.input_layernorm.weight]
Loading weights:  83%|███████████████████████████████████████████████▉          | 300/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.input_layernorm.weight]
Loading weights:  83%|█████████████████████████████████████████████████▊          | 301/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.mlp.down_proj.weight]
Loading weights:  83%|█████████████████████████████████████████████████▊          | 301/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.mlp.down_proj.weight]
Loading weights:  83%|█████████████████████████████████████████████████▉          | 302/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|█████████████████████████████████████████████████▉          | 302/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|███████████████████████████████████████████████████▊          | 303/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  83%|███████████████████████████████████████████████████▊          | 303/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  84%|█████████████████████████████████████████        | 304/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|█████████████████████████████████████████        | 304/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|███████████████████████████████████████████████▉         | 305/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|███████████████████████████████████████████████▉         | 305/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████████████████████████████████████████████         | 306/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████████████████████████████████████████████         | 306/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  85%|████████████████████████████████████████████████▏        | 307/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████████████████████████████████████████████▏        | 307/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████████████████████████████████████████████▎        | 308/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|████████████████████████████████████████████████▎        | 308/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|█████████████████████████████████████████████████▎        | 309/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.input_layernorm.weight]
Loading weights:  85%|█████████████████████████████████████████████████▎        | 309/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.input_layernorm.weight]
Loading weights:  85%|███████████████████████████████████████████████████▏        | 310/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
Loading weights:  85%|███████████████████████████████████████████████████▏        | 310/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
Loading weights:  86%|███████████████████████████████████████████████████▍        | 311/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|███████████████████████████████████████████████████▍        | 311/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|█████████████████████████████████████████████████████▎        | 312/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
Loading weights:  86%|█████████████████████████████████████████████████████▎        | 312/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
Loading weights:  86%|██████████████████████████████████████████▎      | 313/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  86%|██████████████████████████████████████████▎      | 313/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  87%|█████████████████████████████████████████████████▎       | 314/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  87%|█████████████████████████████████████████████████▎       | 314/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  87%|█████████████████████████████████████████████████▍       | 315/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|█████████████████████████████████████████████████▍       | 315/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|█████████████████████████████████████████████████▌       | 316/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|█████████████████████████████████████████████████▌       | 316/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|█████████████████████████████████████████████████▊       | 317/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  87%|█████████████████████████████████████████████████▊       | 317/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  88%|██████████████████████████████████████████████████▊       | 318/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.input_layernorm.weight]
Loading weights:  88%|██████████████████████████████████████████████████▊       | 318/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.input_layernorm.weight]
Loading weights:  88%|████████████████████████████████████████████████████▋       | 319/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████████████████████████████████████████████████▋       | 319/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████████████████████████████████████████████████▉       | 320/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████████████████████████████████████████████████▉       | 320/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|██████████████████████████████████████████████████████▊       | 321/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  88%|██████████████████████████████████████████████████████▊       | 321/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  89%|███████████████████████████████████████████▍     | 322/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  89%|███████████████████████████████████████████▍     | 322/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  89%|██████████████████████████████████████████████████▋      | 323/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|██████████████████████████████████████████████████▋      | 323/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|██████████████████████████████████████████████████▉      | 324/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|██████████████████████████████████████████████████▉      | 324/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  90%|███████████████████████████████████████████████████      | 325/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|███████████████████████████████████████████████████      | 325/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|███████████████████████████████████████████████████▏     | 326/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|███████████████████████████████████████████████████▏     | 326/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|████████████████████████████████████████████████████▏     | 327/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.input_layernorm.weight]
Loading weights:  90%|████████████████████████████████████████████████████▏     | 327/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.input_layernorm.weight]
Loading weights:  90%|██████████████████████████████████████████████████████▏     | 328/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
Loading weights:  90%|██████████████████████████████████████████████████████▏     | 328/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
Loading weights:  91%|██████████████████████████████████████████████████████▍     | 329/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|██████████████████████████████████████████████████████▍     | 329/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|████████████████████████████████████████████████████████▎     | 330/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|████████████████████████████████████████████████████████▎     | 330/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|████████████████████████████████████████████▋    | 331/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|████████████████████████████████████████████▋    | 331/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|████████████████████████████████████████████████████▏    | 332/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  91%|████████████████████████████████████████████████████▏    | 332/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  92%|████████████████████████████████████████████████████▎    | 333/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|████████████████████████████████████████████████████▎    | 333/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|████████████████████████████████████████████████████▍    | 334/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|████████████████████████████████████████████████████▍    | 334/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|████████████████████████████████████████████████████▌    | 335/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  92%|████████████████████████████████████████████████████▌    | 335/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  93%|█████████████████████████████████████████████████████▋    | 336/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.input_layernorm.weight]
Loading weights:  93%|█████████████████████████████████████████████████████▋    | 336/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.input_layernorm.weight]
Loading weights:  93%|███████████████████████████████████████████████████████▋    | 337/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
Loading weights:  93%|███████████████████████████████████████████████████████▋    | 337/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
Loading weights:  93%|███████████████████████████████████████████████████████▊    | 338/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|███████████████████████████████████████████████████████▊    | 338/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████████████████████████████████████████████████████▉    | 339/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
Loading weights:  93%|█████████████████████████████████████████████████████████▉    | 339/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
Loading weights:  94%|█████████████████████████████████████████████▉   | 340/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  94%|█████████████████████████████████████████████▉   | 340/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  94%|█████████████████████████████████████████████████████▌   | 341/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  94%|█████████████████████████████████████████████████████▌   | 341/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  94%|█████████████████████████████████████████████████████▋   | 342/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████████████████████████████████████████████████▋   | 342/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████████████████████████████████████████████████▊   | 343/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  94%|█████████████████████████████████████████████████████▊   | 343/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|██████████████████████████████████████████████████████   | 344/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|██████████████████████████████████████████████████████   | 344/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|███████████████████████████████████████████████████████   | 345/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.input_layernorm.weight]
Loading weights:  95%|███████████████████████████████████████████████████████   | 345/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.input_layernorm.weight]
Loading weights:  95%|█████████████████████████████████████████████████████████▏  | 346/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
Loading weights:  95%|█████████████████████████████████████████████████████████▏  | 346/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
Loading weights:  96%|█████████████████████████████████████████████████████████▎  | 347/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  96%|█████████████████████████████████████████████████████████▎  | 347/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  96%|███████████████████████████████████████████████████████████▍  | 348/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
Loading weights:  96%|███████████████████████████████████████████████████████████▍  | 348/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
Loading weights:  96%|███████████████████████████████████████████████  | 349/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|███████████████████████████████████████████████  | 349/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|██████████████████████████████████████████████████████▉  | 350/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  96%|██████████████████████████████████████████████████████▉  | 350/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  97%|███████████████████████████████████████████████████████  | 351/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|███████████████████████████████████████████████████████  | 351/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|███████████████████████████████████████████████████████▎ | 352/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|███████████████████████████████████████████████████████▎ | 352/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|███████████████████████████████████████████████████████▍ | 353/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  97%|███████████████████████████████████████████████████████▍ | 353/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  98%|████████████████████████████████████████████████████████▌ | 354/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.input_layernorm.weight]
Loading weights:  98%|████████████████████████████████████████████████████████▌ | 354/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.input_layernorm.weight]
Loading weights:  98%|██████████████████████████████████████████████████████████▋ | 355/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
Loading weights:  98%|██████████████████████████████████████████████████████████▋ | 355/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
Loading weights:  98%|██████████████████████████████████████████████████████████▊ | 356/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|██████████████████████████████████████████████████████████▊ | 356/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|████████████████████████████████████████████████████████████▉ | 357/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  98%|████████████████████████████████████████████████████████████▉ | 357/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  99%|████████████████████████████████████████████████▎| 358/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  99%|████████████████████████████████████████████████▎| 358/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  99%|████████████████████████████████████████████████████████▎| 359/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  99%|████████████████████████████████████████████████████████▎| 359/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  99%|████████████████████████████████████████████████████████▌| 360/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|████████████████████████████████████████████████████████▌| 360/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|████████████████████████████████████████████████████████▋| 361/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights:  99%|████████████████████████████████████████████████████████▋| 361/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|████████████████████████████████████████████████████████▊| 362/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|████████████████████████████████████████████████████████▊| 362/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|███████████████████████████████████████████████████████████████████████████████| 363/363 [00:00<00:00, 2182.81it/s, Materializing param=model.norm.weight]
Loading weights: 100%|███████████████████████████████████████████████████████████████████████████████| 363/363 [00:00<00:00, 2182.81it/s, Materializing param=model.norm.weight]
Loading weights: 100%|███████████████████████████████████████████████████████████████████████████████| 363/363 [00:00<00:00, 2040.45it/s, Materializing param=model.norm.weight]
[2026-02-06 15:51:29,234] [WARNING] [accelerate.utils.dataclasses.__post_init__:1962] [PID:7207] sync_module_states is obsolete in FSDP2, as it is not needed anymore.Setting sync_module_states to None.Multiple deprecation warnings due to FSDP2 conversion:
sharding_strategy is deprecated in favor of reshard_after_forward. This will be removed in a future version of Accelerate.
[2026-02-06 15:51:34,875] [WARNING] [py.warnings._showwarnmsg:110] [PID:7207] /root/axolotl/.venv/lib/python3.11/site-packages/torch/distributed/distributed_c10d.py:4807: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
  warnings.warn(  # warn only once