MVP/distill.log

The fast path is not available because one of the required library is not installed. Falling back to torch implementation. To install follow https://github.com/fla-org/flash-linear-attention#installation and https://github.com/Dao-AILab/causal-conv1d
Loading teacher from /home/void0x14/.cache/huggingface/hub/models--Qwen--Qwen3.5-0.8B-Base/snapshots/dc7cdfe2ee4154fa7e30f5b51ca41bfa40174e68...

Loading weights:   0%|          | 0/473 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/473 [00:00<00:00, 7371.36it/s, Materializing param=language_model.embed_tokens.weight]
Loading weights:   0%|          | 1/473 [00:00<00:00, 3130.08it/s, Materializing param=language_model.embed_tokens.weight]
Loading weights:   0%|          | 2/473 [00:00<00:59,  7.93it/s, Materializing param=language_model.embed_tokens.weight]  
Loading weights:   0%|          | 2/473 [00:00<00:59,  7.93it/s, Materializing param=language_model.layers.0.input_layernorm.weight]
Loading weights:   0%|          | 2/473 [00:00<00:59,  7.93it/s, Materializing param=language_model.layers.0.input_layernorm.weight]
Loading weights:   1%|          | 3/473 [00:00<00:59,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.A_log]     
Loading weights:   1%|          | 3/473 [00:00<00:59,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.A_log]
Loading weights:   1%|          | 4/473 [00:00<00:59,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.conv1d.weight]
Loading weights:   1%|          | 4/473 [00:00<00:59,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.conv1d.weight]
Loading weights:   1%|          | 5/473 [00:00<00:59,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.dt_bias]      
Loading weights:   1%|          | 5/473 [00:00<00:59,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.dt_bias]
Loading weights:   1%|▏         | 6/473 [00:00<00:58,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.in_proj_a.weight]
Loading weights:   1%|▏         | 6/473 [00:00<00:58,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.in_proj_a.weight]
Loading weights:   1%|▏         | 7/473 [00:00<00:58,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.in_proj_b.weight]
Loading weights:   1%|▏         | 7/473 [00:00<00:58,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.in_proj_b.weight]
Loading weights:   2%|▏         | 8/473 [00:00<00:58,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.in_proj_qkv.weight]
Loading weights:   2%|▏         | 8/473 [00:00<00:58,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.in_proj_qkv.weight]
Loading weights:   2%|▏         | 9/473 [00:00<00:58,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.in_proj_z.weight]  
Loading weights:   2%|▏         | 9/473 [00:00<00:58,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.in_proj_z.weight]
Loading weights:   2%|▏         | 10/473 [00:00<00:58,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.norm.weight]    
Loading weights:   2%|▏         | 10/473 [00:00<00:58,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.norm.weight]
Loading weights:   2%|▏         | 11/473 [00:00<00:58,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.out_proj.weight]
Loading weights:   2%|▏         | 11/473 [00:00<00:58,  7.93it/s, Materializing param=language_model.layers.0.linear_attn.out_proj.weight]
Loading weights:   3%|▎         | 12/473 [00:00<00:58,  7.93it/s, Materializing param=language_model.layers.0.mlp.down_proj.weight]       
Loading weights:   3%|▎         | 12/473 [00:00<00:58,  7.93it/s, Materializing param=language_model.layers.0.mlp.down_proj.weight]
Loading weights:   3%|▎         | 13/473 [00:00<00:58,  7.93it/s, Materializing param=language_model.layers.0.mlp.gate_proj.weight]
Loading weights:   3%|▎         | 13/473 [00:00<00:58,  7.93it/s, Materializing param=language_model.layers.0.mlp.gate_proj.weight]
Loading weights:   3%|▎         | 14/473 [00:00<00:57,  7.93it/s, Materializing param=language_model.layers.0.mlp.up_proj.weight]  
Loading weights:   3%|▎         | 14/473 [00:00<00:57,  7.93it/s, Materializing param=language_model.layers.0.mlp.up_proj.weight]
Loading weights:   3%|▎         | 15/473 [00:00<00:57,  7.93it/s, Materializing param=language_model.layers.0.post_attention_layernorm.weight]
Loading weights:   3%|▎         | 15/473 [00:00<00:57,  7.93it/s, Materializing param=language_model.layers.0.post_attention_layernorm.weight]
Loading weights:   3%|▎         | 16/473 [00:00<00:57,  7.93it/s, Materializing param=language_model.layers.1.input_layernorm.weight]         
Loading weights:   3%|▎         | 16/473 [00:00<00:57,  7.93it/s, Materializing param=language_model.layers.1.input_layernorm.weight]
Loading weights:   4%|▎         | 17/473 [00:00<00:57,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.A_log]     
Loading weights:   4%|▎         | 17/473 [00:00<00:57,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.A_log]
Loading weights:   4%|▍         | 18/473 [00:00<00:57,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.conv1d.weight]
Loading weights:   4%|▍         | 18/473 [00:00<00:57,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.conv1d.weight]
Loading weights:   4%|▍         | 19/473 [00:00<00:57,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.dt_bias]      
Loading weights:   4%|▍         | 19/473 [00:00<00:57,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.dt_bias]
Loading weights:   4%|▍         | 20/473 [00:00<00:57,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.in_proj_a.weight]
Loading weights:   4%|▍         | 20/473 [00:00<00:57,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.in_proj_a.weight]
Loading weights:   4%|▍         | 21/473 [00:00<00:57,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.in_proj_b.weight]
Loading weights:   4%|▍         | 21/473 [00:00<00:57,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.in_proj_b.weight]
Loading weights:   5%|▍         | 22/473 [00:00<00:56,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.in_proj_qkv.weight]
Loading weights:   5%|▍         | 22/473 [00:00<00:56,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.in_proj_qkv.weight]
Loading weights:   5%|▍         | 23/473 [00:00<00:56,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.in_proj_z.weight]  
Loading weights:   5%|▍         | 23/473 [00:00<00:56,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.in_proj_z.weight]
Loading weights:   5%|▌         | 24/473 [00:00<00:56,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.norm.weight]     
Loading weights:   5%|▌         | 24/473 [00:00<00:56,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.norm.weight]
Loading weights:   5%|▌         | 25/473 [00:00<00:56,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.out_proj.weight]
Loading weights:   5%|▌         | 25/473 [00:00<00:56,  7.93it/s, Materializing param=language_model.layers.1.linear_attn.out_proj.weight]
Loading weights:   5%|▌         | 26/473 [00:00<00:56,  7.93it/s, Materializing param=language_model.layers.1.mlp.down_proj.weight]       
Loading weights:   5%|▌         | 26/473 [00:00<00:56,  7.93it/s, Materializing param=language_model.layers.1.mlp.down_proj.weight]
Loading weights:   6%|▌         | 27/473 [00:00<00:56,  7.93it/s, Materializing param=language_model.layers.1.mlp.gate_proj.weight]
Loading weights:   6%|▌         | 27/473 [00:00<00:56,  7.93it/s, Materializing param=language_model.layers.1.mlp.gate_proj.weight]
Loading weights:   6%|▌         | 28/473 [00:00<00:56,  7.93it/s, Materializing param=language_model.layers.1.mlp.up_proj.weight]  
Loading weights:   6%|▌         | 28/473 [00:00<00:56,  7.93it/s, Materializing param=language_model.layers.1.mlp.up_proj.weight]
Loading weights:   6%|▌         | 29/473 [00:00<00:56,  7.93it/s, Materializing param=language_model.layers.1.post_attention_layernorm.weight]
Loading weights:   6%|▌         | 29/473 [00:00<00:56,  7.93it/s, Materializing param=language_model.layers.1.post_attention_layernorm.weight]
Loading weights:   6%|▋         | 30/473 [00:00<00:55,  7.93it/s, Materializing param=language_model.layers.2.input_layernorm.weight]         
Loading weights:   6%|▋         | 30/473 [00:00<00:55,  7.93it/s, Materializing param=language_model.layers.2.input_layernorm.weight]
Loading weights:   7%|▋         | 31/473 [00:00<00:55,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.A_log]     
Loading weights:   7%|▋         | 31/473 [00:00<00:55,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.A_log]
Loading weights:   7%|▋         | 32/473 [00:00<00:55,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.conv1d.weight]
Loading weights:   7%|▋         | 32/473 [00:00<00:55,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.conv1d.weight]
Loading weights:   7%|▋         | 33/473 [00:00<00:55,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.dt_bias]      
Loading weights:   7%|▋         | 33/473 [00:00<00:55,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.dt_bias]
Loading weights:   7%|▋         | 34/473 [00:00<00:55,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.in_proj_a.weight]
Loading weights:   7%|▋         | 34/473 [00:00<00:55,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.in_proj_a.weight]
Loading weights:   7%|▋         | 35/473 [00:00<00:55,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.in_proj_b.weight]
Loading weights:   7%|▋         | 35/473 [00:00<00:55,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.in_proj_b.weight]
Loading weights:   8%|▊         | 36/473 [00:00<00:55,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.in_proj_qkv.weight]
Loading weights:   8%|▊         | 36/473 [00:00<00:55,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.in_proj_qkv.weight]
Loading weights:   8%|▊         | 37/473 [00:00<00:55,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.in_proj_z.weight]  
Loading weights:   8%|▊         | 37/473 [00:00<00:55,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.in_proj_z.weight]
Loading weights:   8%|▊         | 38/473 [00:00<00:54,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.norm.weight]     
Loading weights:   8%|▊         | 38/473 [00:00<00:54,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.norm.weight]
Loading weights:   8%|▊         | 39/473 [00:00<00:54,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.out_proj.weight]
Loading weights:   8%|▊         | 39/473 [00:00<00:54,  7.93it/s, Materializing param=language_model.layers.2.linear_attn.out_proj.weight]
Loading weights:   8%|▊         | 40/473 [00:00<00:54,  7.93it/s, Materializing param=language_model.layers.2.mlp.down_proj.weight]       
Loading weights:   8%|▊         | 40/473 [00:00<00:54,  7.93it/s, Materializing param=language_model.layers.2.mlp.down_proj.weight]
Loading weights:   9%|▊         | 41/473 [00:00<00:54,  7.93it/s, Materializing param=language_model.layers.2.mlp.gate_proj.weight]
Loading weights:   9%|▊         | 41/473 [00:00<00:54,  7.93it/s, Materializing param=language_model.layers.2.mlp.gate_proj.weight]
Loading weights:   9%|▉         | 42/473 [00:00<00:54,  7.93it/s, Materializing param=language_model.layers.2.mlp.up_proj.weight]  
Loading weights:   9%|▉         | 42/473 [00:00<00:54,  7.93it/s, Materializing param=language_model.layers.2.mlp.up_proj.weight]
Loading weights:   9%|▉         | 43/473 [00:00<00:54,  7.93it/s, Materializing param=language_model.layers.2.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 43/473 [00:00<00:54,  7.93it/s, Materializing param=language_model.layers.2.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 44/473 [00:00<00:54,  7.93it/s, Materializing param=language_model.layers.3.input_layernorm.weight]         
Loading weights:   9%|▉         | 44/473 [00:00<00:54,  7.93it/s, Materializing param=language_model.layers.3.input_layernorm.weight]
Loading weights:  10%|▉         | 45/473 [00:00<00:53,  7.93it/s, Materializing param=language_model.layers.3.mlp.down_proj.weight]  
Loading weights:  10%|▉         | 45/473 [00:00<00:53,  7.93it/s, Materializing param=language_model.layers.3.mlp.down_proj.weight]
Loading weights:  10%|▉         | 46/473 [00:00<00:53,  7.93it/s, Materializing param=language_model.layers.3.mlp.gate_proj.weight]
Loading weights:  10%|▉         | 46/473 [00:00<00:53,  7.93it/s, Materializing param=language_model.layers.3.mlp.gate_proj.weight]
Loading weights:  10%|▉         | 47/473 [00:00<00:53,  7.93it/s, Materializing param=language_model.layers.3.mlp.up_proj.weight]  
Loading weights:  10%|▉         | 47/473 [00:00<00:53,  7.93it/s, Materializing param=language_model.layers.3.mlp.up_proj.weight]
Loading weights:  10%|█         | 48/473 [00:00<00:53,  7.93it/s, Materializing param=language_model.layers.3.post_attention_layernorm.weight]
Loading weights:  10%|█         | 48/473 [00:00<00:53,  7.93it/s, Materializing param=language_model.layers.3.post_attention_layernorm.weight]
Loading weights:  10%|█         | 49/473 [00:00<00:53,  7.93it/s, Materializing param=language_model.layers.3.self_attn.k_norm.weight]        
Loading weights:  10%|█         | 49/473 [00:00<00:53,  7.93it/s, Materializing param=language_model.layers.3.self_attn.k_norm.weight]
Loading weights:  11%|█         | 50/473 [00:00<00:53,  7.93it/s, Materializing param=language_model.layers.3.self_attn.k_proj.weight]
Loading weights:  11%|█         | 50/473 [00:00<00:53,  7.93it/s, Materializing param=language_model.layers.3.self_attn.k_proj.weight]
Loading weights:  11%|█         | 51/473 [00:00<00:53,  7.93it/s, Materializing param=language_model.layers.3.self_attn.o_proj.weight]
Loading weights:  11%|█         | 51/473 [00:00<00:53,  7.93it/s, Materializing param=language_model.layers.3.self_attn.o_proj.weight]
Loading weights:  11%|█         | 52/473 [00:00<00:53,  7.93it/s, Materializing param=language_model.layers.3.self_attn.q_norm.weight]
Loading weights:  11%|█         | 52/473 [00:00<00:53,  7.93it/s, Materializing param=language_model.layers.3.self_attn.q_norm.weight]
Loading weights:  11%|█         | 53/473 [00:00<00:52,  7.93it/s, Materializing param=language_model.layers.3.self_attn.q_proj.weight]
Loading weights:  11%|█         | 53/473 [00:00<00:52,  7.93it/s, Materializing param=language_model.layers.3.self_attn.q_proj.weight]
Loading weights:  11%|█▏        | 54/473 [00:00<00:52,  7.93it/s, Materializing param=language_model.layers.3.self_attn.v_proj.weight]
Loading weights:  11%|█▏        | 54/473 [00:00<00:52,  7.93it/s, Materializing param=language_model.layers.3.self_attn.v_proj.weight]
Loading weights:  12%|█▏        | 55/473 [00:00<00:52,  7.93it/s, Materializing param=language_model.layers.4.input_layernorm.weight] 
Loading weights:  12%|█▏        | 55/473 [00:00<00:52,  7.93it/s, Materializing param=language_model.layers.4.input_layernorm.weight]
Loading weights:  12%|█▏        | 56/473 [00:00<00:52,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.A_log]     
Loading weights:  12%|█▏        | 56/473 [00:00<00:52,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.A_log]
Loading weights:  12%|█▏        | 57/473 [00:00<00:52,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.conv1d.weight]
Loading weights:  12%|█▏        | 57/473 [00:00<00:52,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.conv1d.weight]
Loading weights:  12%|█▏        | 58/473 [00:00<00:52,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.dt_bias]      
Loading weights:  12%|█▏        | 58/473 [00:00<00:52,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.dt_bias]
Loading weights:  12%|█▏        | 59/473 [00:00<00:52,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.in_proj_a.weight]
Loading weights:  12%|█▏        | 59/473 [00:00<00:52,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.in_proj_a.weight]
Loading weights:  13%|█▎        | 60/473 [00:00<00:52,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.in_proj_b.weight]
Loading weights:  13%|█▎        | 60/473 [00:00<00:52,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.in_proj_b.weight]
Loading weights:  13%|█▎        | 61/473 [00:00<00:51,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.in_proj_qkv.weight]
Loading weights:  13%|█▎        | 61/473 [00:00<00:51,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.in_proj_qkv.weight]
Loading weights:  13%|█▎        | 62/473 [00:00<00:51,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.in_proj_z.weight]  
Loading weights:  13%|█▎        | 62/473 [00:00<00:51,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.in_proj_z.weight]
Loading weights:  13%|█▎        | 63/473 [00:00<00:51,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.norm.weight]     
Loading weights:  13%|█▎        | 63/473 [00:00<00:51,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.norm.weight]
Loading weights:  14%|█▎        | 64/473 [00:00<00:51,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.out_proj.weight]
Loading weights:  14%|█▎        | 64/473 [00:00<00:51,  7.93it/s, Materializing param=language_model.layers.4.linear_attn.out_proj.weight]
Loading weights:  14%|█▎        | 65/473 [00:00<00:51,  7.93it/s, Materializing param=language_model.layers.4.mlp.down_proj.weight]       
Loading weights:  14%|█▎        | 65/473 [00:00<00:51,  7.93it/s, Materializing param=language_model.layers.4.mlp.down_proj.weight]
Loading weights:  14%|█▍        | 66/473 [00:00<00:51,  7.93it/s, Materializing param=language_model.layers.4.mlp.gate_proj.weight]
Loading weights:  14%|█▍        | 66/473 [00:00<00:51,  7.93it/s, Materializing param=language_model.layers.4.mlp.gate_proj.weight]
Loading weights:  14%|█▍        | 67/473 [00:00<00:51,  7.93it/s, Materializing param=language_model.layers.4.mlp.up_proj.weight]  
Loading weights:  14%|█▍        | 67/473 [00:00<00:51,  7.93it/s, Materializing param=language_model.layers.4.mlp.up_proj.weight]
Loading weights:  14%|█▍        | 68/473 [00:00<00:51,  7.93it/s, Materializing param=language_model.layers.4.post_attention_layernorm.weight]
Loading weights:  14%|█▍        | 68/473 [00:00<00:51,  7.93it/s, Materializing param=language_model.layers.4.post_attention_layernorm.weight]
Loading weights:  15%|█▍        | 69/473 [00:00<00:50,  7.93it/s, Materializing param=language_model.layers.5.input_layernorm.weight]         
Loading weights:  15%|█▍        | 69/473 [00:00<00:50,  7.93it/s, Materializing param=language_model.layers.5.input_layernorm.weight]
Loading weights:  15%|█▍        | 70/473 [00:00<00:50,  7.93it/s, Materializing param=language_model.layers.5.linear_attn.A_log]     
Loading weights:  15%|█▍        | 70/473 [00:00<00:50,  7.93it/s, Materializing param=language_model.layers.5.linear_attn.A_log]
Loading weights:  15%|█▌        | 71/473 [00:00<00:50,  7.93it/s, Materializing param=language_model.layers.5.linear_attn.conv1d.weight]
Loading weights:  15%|█▌        | 71/473 [00:00<00:50,  7.93it/s, Materializing param=language_model.layers.5.linear_attn.conv1d.weight]
Loading weights:  15%|█▌        | 72/473 [00:00<00:50,  7.93it/s, Materializing param=language_model.layers.5.linear_attn.dt_bias]      
Loading weights:  15%|█▌        | 72/473 [00:00<00:50,  7.93it/s, Materializing param=language_model.layers.5.linear_attn.dt_bias]
Loading weights:  15%|█▌        | 73/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.linear_attn.dt_bias]
Loading weights:  15%|█▌        | 73/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.linear_attn.in_proj_a.weight]
Loading weights:  15%|█▌        | 73/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.linear_attn.in_proj_a.weight]
Loading weights:  16%|█▌        | 74/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.linear_attn.in_proj_b.weight]
Loading weights:  16%|█▌        | 74/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.linear_attn.in_proj_b.weight]
Loading weights:  16%|█▌        | 75/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.linear_attn.in_proj_qkv.weight]
Loading weights:  16%|█▌        | 75/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.linear_attn.in_proj_qkv.weight]
Loading weights:  16%|█▌        | 76/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.linear_attn.in_proj_z.weight]  
Loading weights:  16%|█▌        | 76/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.linear_attn.in_proj_z.weight]
Loading weights:  16%|█▋        | 77/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.linear_attn.norm.weight]     
Loading weights:  16%|█▋        | 77/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.linear_attn.norm.weight]
Loading weights:  16%|█▋        | 78/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.linear_attn.out_proj.weight]
Loading weights:  16%|█▋        | 78/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.linear_attn.out_proj.weight]
Loading weights:  17%|█▋        | 79/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.mlp.down_proj.weight]       
Loading weights:  17%|█▋        | 79/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.mlp.down_proj.weight]
Loading weights:  17%|█▋        | 80/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.mlp.gate_proj.weight]
Loading weights:  17%|█▋        | 80/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.mlp.gate_proj.weight]
Loading weights:  17%|█▋        | 81/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.mlp.up_proj.weight]  
Loading weights:  17%|█▋        | 81/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.mlp.up_proj.weight]
Loading weights:  17%|█▋        | 82/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.post_attention_layernorm.weight]
Loading weights:  17%|█▋        | 82/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.5.post_attention_layernorm.weight]
Loading weights:  18%|█▊        | 83/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.input_layernorm.weight]         
Loading weights:  18%|█▊        | 83/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.input_layernorm.weight]
Loading weights:  18%|█▊        | 84/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.A_log]     
Loading weights:  18%|█▊        | 84/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.A_log]
Loading weights:  18%|█▊        | 85/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.conv1d.weight]
Loading weights:  18%|█▊        | 85/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.conv1d.weight]
Loading weights:  18%|█▊        | 86/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.dt_bias]      
Loading weights:  18%|█▊        | 86/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.dt_bias]
Loading weights:  18%|█▊        | 87/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.in_proj_a.weight]
Loading weights:  18%|█▊        | 87/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.in_proj_a.weight]
Loading weights:  19%|█▊        | 88/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.in_proj_b.weight]
Loading weights:  19%|█▊        | 88/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.in_proj_b.weight]
Loading weights:  19%|█▉        | 89/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.in_proj_qkv.weight]
Loading weights:  19%|█▉        | 89/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.in_proj_qkv.weight]
Loading weights:  19%|█▉        | 90/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.in_proj_z.weight]  
Loading weights:  19%|█▉        | 90/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.in_proj_z.weight]
Loading weights:  19%|█▉        | 91/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.norm.weight]     
Loading weights:  19%|█▉        | 91/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.norm.weight]
Loading weights:  19%|█▉        | 92/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.out_proj.weight]
Loading weights:  19%|█▉        | 92/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.linear_attn.out_proj.weight]
Loading weights:  20%|█▉        | 93/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.mlp.down_proj.weight]       
Loading weights:  20%|█▉        | 93/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.mlp.down_proj.weight]
Loading weights:  20%|█▉        | 94/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.mlp.gate_proj.weight]
Loading weights:  20%|█▉        | 94/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.mlp.gate_proj.weight]
Loading weights:  20%|██        | 95/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.mlp.up_proj.weight]  
Loading weights:  20%|██        | 95/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.mlp.up_proj.weight]
Loading weights:  20%|██        | 96/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.post_attention_layernorm.weight]
Loading weights:  20%|██        | 96/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.6.post_attention_layernorm.weight]
Loading weights:  21%|██        | 97/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.input_layernorm.weight]         
Loading weights:  21%|██        | 97/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.input_layernorm.weight]
Loading weights:  21%|██        | 98/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.mlp.down_proj.weight]  
Loading weights:  21%|██        | 98/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.mlp.down_proj.weight]
Loading weights:  21%|██        | 99/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.mlp.gate_proj.weight]
Loading weights:  21%|██        | 99/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.mlp.gate_proj.weight]
Loading weights:  21%|██        | 100/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.mlp.up_proj.weight] 
Loading weights:  21%|██        | 100/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.mlp.up_proj.weight]
Loading weights:  21%|██▏       | 101/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.post_attention_layernorm.weight]
Loading weights:  21%|██▏       | 101/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.post_attention_layernorm.weight]
Loading weights:  22%|██▏       | 102/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.self_attn.k_norm.weight]        
Loading weights:  22%|██▏       | 102/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.self_attn.k_norm.weight]
Loading weights:  22%|██▏       | 103/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 103/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 104/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 104/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 105/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.self_attn.q_norm.weight]
Loading weights:  22%|██▏       | 105/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.self_attn.q_norm.weight]
Loading weights:  22%|██▏       | 106/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.self_attn.q_proj.weight]
Loading weights:  22%|██▏       | 106/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 107/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 107/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.7.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 108/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.8.input_layernorm.weight] 
Loading weights:  23%|██▎       | 108/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.8.input_layernorm.weight]
Loading weights:  23%|██▎       | 109/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.8.linear_attn.A_log]     
Loading weights:  23%|██▎       | 109/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.8.linear_attn.A_log]
Loading weights:  23%|██▎       | 110/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.8.linear_attn.conv1d.weight]
Loading weights:  23%|██▎       | 110/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.8.linear_attn.conv1d.weight]
Loading weights:  23%|██▎       | 111/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.8.linear_attn.dt_bias]      
Loading weights:  23%|██▎       | 111/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.8.linear_attn.dt_bias]
Loading weights:  24%|██▎       | 112/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.8.linear_attn.in_proj_a.weight]
Loading weights:  24%|██▎       | 112/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.8.linear_attn.in_proj_a.weight]
Loading weights:  24%|██▍       | 113/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.8.linear_attn.in_proj_b.weight]
Loading weights:  24%|██▍       | 113/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.8.linear_attn.in_proj_b.weight]
Loading weights:  24%|██▍       | 114/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.8.linear_attn.in_proj_qkv.weight]
Loading weights:  24%|██▍       | 114/473 [00:00<00:01, 260.07it/s, Materializing param=language_model.layers.8.linear_attn.in_proj_qkv.weight]
Loading weights:  24%|██▍       | 115/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.8.linear_attn.in_proj_qkv.weight]
Loading weights:  24%|██▍       | 115/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.8.linear_attn.in_proj_z.weight]  
Loading weights:  24%|██▍       | 115/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.8.linear_attn.in_proj_z.weight]
Loading weights:  25%|██▍       | 116/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.8.linear_attn.norm.weight]     
Loading weights:  25%|██▍       | 116/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.8.linear_attn.norm.weight]
Loading weights:  25%|██▍       | 117/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.8.linear_attn.out_proj.weight]
Loading weights:  25%|██▍       | 117/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.8.linear_attn.out_proj.weight]
Loading weights:  25%|██▍       | 118/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.8.mlp.down_proj.weight]       
Loading weights:  25%|██▍       | 118/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.8.mlp.down_proj.weight]
Loading weights:  25%|██▌       | 119/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.8.mlp.gate_proj.weight]
Loading weights:  25%|██▌       | 119/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.8.mlp.gate_proj.weight]
Loading weights:  25%|██▌       | 120/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.8.mlp.up_proj.weight]  
Loading weights:  25%|██▌       | 120/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.8.mlp.up_proj.weight]
Loading weights:  26%|██▌       | 121/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.8.post_attention_layernorm.weight]
Loading weights:  26%|██▌       | 121/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.8.post_attention_layernorm.weight]
Loading weights:  26%|██▌       | 122/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.input_layernorm.weight]         
Loading weights:  26%|██▌       | 122/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.input_layernorm.weight]
Loading weights:  26%|██▌       | 123/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.A_log]     
Loading weights:  26%|██▌       | 123/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.A_log]
Loading weights:  26%|██▌       | 124/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.conv1d.weight]
Loading weights:  26%|██▌       | 124/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.conv1d.weight]
Loading weights:  26%|██▋       | 125/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.dt_bias]      
Loading weights:  26%|██▋       | 125/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.dt_bias]
Loading weights:  27%|██▋       | 126/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.in_proj_a.weight]
Loading weights:  27%|██▋       | 126/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.in_proj_a.weight]
Loading weights:  27%|██▋       | 127/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.in_proj_b.weight]
Loading weights:  27%|██▋       | 127/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.in_proj_b.weight]
Loading weights:  27%|██▋       | 128/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.in_proj_qkv.weight]
Loading weights:  27%|██▋       | 128/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.in_proj_qkv.weight]
Loading weights:  27%|██▋       | 129/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.in_proj_z.weight]  
Loading weights:  27%|██▋       | 129/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.in_proj_z.weight]
Loading weights:  27%|██▋       | 130/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.norm.weight]     
Loading weights:  27%|██▋       | 130/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.norm.weight]
Loading weights:  28%|██▊       | 131/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.out_proj.weight]
Loading weights:  28%|██▊       | 131/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.linear_attn.out_proj.weight]
Loading weights:  28%|██▊       | 132/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.mlp.down_proj.weight]       
Loading weights:  28%|██▊       | 132/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.mlp.down_proj.weight]
Loading weights:  28%|██▊       | 133/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.mlp.gate_proj.weight]
Loading weights:  28%|██▊       | 133/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.mlp.gate_proj.weight]
Loading weights:  28%|██▊       | 134/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.mlp.up_proj.weight]  
Loading weights:  28%|██▊       | 134/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.mlp.up_proj.weight]
Loading weights:  29%|██▊       | 135/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.post_attention_layernorm.weight]
Loading weights:  29%|██▊       | 135/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.9.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 136/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.input_layernorm.weight]        
Loading weights:  29%|██▉       | 136/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.input_layernorm.weight]
Loading weights:  29%|██▉       | 137/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.A_log]     
Loading weights:  29%|██▉       | 137/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.A_log]
Loading weights:  29%|██▉       | 138/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.conv1d.weight]
Loading weights:  29%|██▉       | 138/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.conv1d.weight]
Loading weights:  29%|██▉       | 139/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.dt_bias]      
Loading weights:  29%|██▉       | 139/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.dt_bias]
Loading weights:  30%|██▉       | 140/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.in_proj_a.weight]
Loading weights:  30%|██▉       | 140/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.in_proj_a.weight]
Loading weights:  30%|██▉       | 141/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.in_proj_b.weight]
Loading weights:  30%|██▉       | 141/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.in_proj_b.weight]
Loading weights:  30%|███       | 142/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.in_proj_qkv.weight]
Loading weights:  30%|███       | 142/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.in_proj_qkv.weight]
Loading weights:  30%|███       | 143/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.in_proj_z.weight]  
Loading weights:  30%|███       | 143/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.in_proj_z.weight]
Loading weights:  30%|███       | 144/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.norm.weight]     
Loading weights:  30%|███       | 144/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.norm.weight]
Loading weights:  31%|███       | 145/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.out_proj.weight]
Loading weights:  31%|███       | 145/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.linear_attn.out_proj.weight]
Loading weights:  31%|███       | 146/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.mlp.down_proj.weight]       
Loading weights:  31%|███       | 146/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.mlp.down_proj.weight]
Loading weights:  31%|███       | 147/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.mlp.gate_proj.weight]
Loading weights:  31%|███       | 147/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.mlp.gate_proj.weight]
Loading weights:  31%|███▏      | 148/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.mlp.up_proj.weight]  
Loading weights:  31%|███▏      | 148/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.mlp.up_proj.weight]
Loading weights:  32%|███▏      | 149/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.post_attention_layernorm.weight]
Loading weights:  32%|███▏      | 149/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.10.post_attention_layernorm.weight]
Loading weights:  32%|███▏      | 150/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.11.input_layernorm.weight]         
Loading weights:  32%|███▏      | 150/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.11.input_layernorm.weight]
Loading weights:  32%|███▏      | 151/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.11.mlp.down_proj.weight]  
Loading weights:  32%|███▏      | 151/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.11.mlp.down_proj.weight]
Loading weights:  32%|███▏      | 152/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.11.mlp.gate_proj.weight]
Loading weights:  32%|███▏      | 152/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.11.mlp.gate_proj.weight]
Loading weights:  32%|███▏      | 153/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.11.mlp.up_proj.weight]  
Loading weights:  32%|███▏      | 153/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.11.mlp.up_proj.weight]
Loading weights:  33%|███▎      | 154/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.11.post_attention_layernorm.weight]
Loading weights:  33%|███▎      | 154/473 [00:00<00:01, 287.24it/s, Materializing param=language_model.layers.11.post_attention_layernorm.weight]
Loading weights:  33%|███▎      | 155/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.11.post_attention_layernorm.weight]
Loading weights:  33%|███▎      | 155/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.11.self_attn.k_norm.weight]        
Loading weights:  33%|███▎      | 155/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.11.self_attn.k_norm.weight]
Loading weights:  33%|███▎      | 156/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.11.self_attn.k_proj.weight]
Loading weights:  33%|███▎      | 156/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.11.self_attn.k_proj.weight]
Loading weights:  33%|███▎      | 157/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.11.self_attn.o_proj.weight]
Loading weights:  33%|███▎      | 157/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.11.self_attn.o_proj.weight]
Loading weights:  33%|███▎      | 158/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.11.self_attn.q_norm.weight]
Loading weights:  33%|███▎      | 158/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.11.self_attn.q_norm.weight]
Loading weights:  34%|███▎      | 159/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.11.self_attn.q_proj.weight]
Loading weights:  34%|███▎      | 159/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.11.self_attn.q_proj.weight]
Loading weights:  34%|███▍      | 160/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.11.self_attn.v_proj.weight]
Loading weights:  34%|███▍      | 160/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.11.self_attn.v_proj.weight]
Loading weights:  34%|███▍      | 161/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.input_layernorm.weight] 
Loading weights:  34%|███▍      | 161/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.input_layernorm.weight]
Loading weights:  34%|███▍      | 162/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.A_log]     
Loading weights:  34%|███▍      | 162/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.A_log]
Loading weights:  34%|███▍      | 163/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.conv1d.weight]
Loading weights:  34%|███▍      | 163/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.conv1d.weight]
Loading weights:  35%|███▍      | 164/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.dt_bias]      
Loading weights:  35%|███▍      | 164/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.dt_bias]
Loading weights:  35%|███▍      | 165/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.in_proj_a.weight]
Loading weights:  35%|███▍      | 165/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.in_proj_a.weight]
Loading weights:  35%|███▌      | 166/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.in_proj_b.weight]
Loading weights:  35%|███▌      | 166/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.in_proj_b.weight]
Loading weights:  35%|███▌      | 167/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.in_proj_qkv.weight]
Loading weights:  35%|███▌      | 167/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.in_proj_qkv.weight]
Loading weights:  36%|███▌      | 168/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.in_proj_z.weight]  
Loading weights:  36%|███▌      | 168/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.in_proj_z.weight]
Loading weights:  36%|███▌      | 169/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.norm.weight]     
Loading weights:  36%|███▌      | 169/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.norm.weight]
Loading weights:  36%|███▌      | 170/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.out_proj.weight]
Loading weights:  36%|███▌      | 170/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.linear_attn.out_proj.weight]
Loading weights:  36%|███▌      | 171/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.mlp.down_proj.weight]       
Loading weights:  36%|███▌      | 171/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.mlp.down_proj.weight]
Loading weights:  36%|███▋      | 172/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.mlp.gate_proj.weight]
Loading weights:  36%|███▋      | 172/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.mlp.gate_proj.weight]
Loading weights:  37%|███▋      | 173/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.mlp.up_proj.weight]  
Loading weights:  37%|███▋      | 173/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.mlp.up_proj.weight]
Loading weights:  37%|███▋      | 174/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.post_attention_layernorm.weight]
Loading weights:  37%|███▋      | 174/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.12.post_attention_layernorm.weight]
Loading weights:  37%|███▋      | 175/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.input_layernorm.weight]         
Loading weights:  37%|███▋      | 175/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.input_layernorm.weight]
Loading weights:  37%|███▋      | 176/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.A_log]     
Loading weights:  37%|███▋      | 176/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.A_log]
Loading weights:  37%|███▋      | 177/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.conv1d.weight]
Loading weights:  37%|███▋      | 177/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.conv1d.weight]
Loading weights:  38%|███▊      | 178/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.dt_bias]      
Loading weights:  38%|███▊      | 178/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.dt_bias]
Loading weights:  38%|███▊      | 179/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.in_proj_a.weight]
Loading weights:  38%|███▊      | 179/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.in_proj_a.weight]
Loading weights:  38%|███▊      | 180/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.in_proj_b.weight]
Loading weights:  38%|███▊      | 180/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.in_proj_b.weight]
Loading weights:  38%|███▊      | 181/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.in_proj_qkv.weight]
Loading weights:  38%|███▊      | 181/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.in_proj_qkv.weight]
Loading weights:  38%|███▊      | 182/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.in_proj_z.weight]  
Loading weights:  38%|███▊      | 182/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.in_proj_z.weight]
Loading weights:  39%|███▊      | 183/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.norm.weight]     
Loading weights:  39%|███▊      | 183/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.norm.weight]
Loading weights:  39%|███▉      | 184/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.out_proj.weight]
Loading weights:  39%|███▉      | 184/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.linear_attn.out_proj.weight]
Loading weights:  39%|███▉      | 185/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.mlp.down_proj.weight]       
Loading weights:  39%|███▉      | 185/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.mlp.down_proj.weight]
Loading weights:  39%|███▉      | 186/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.mlp.gate_proj.weight]
Loading weights:  39%|███▉      | 186/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.mlp.gate_proj.weight]
Loading weights:  40%|███▉      | 187/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.mlp.up_proj.weight]  
Loading weights:  40%|███▉      | 187/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.mlp.up_proj.weight]
Loading weights:  40%|███▉      | 188/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.post_attention_layernorm.weight]
Loading weights:  40%|███▉      | 188/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.13.post_attention_layernorm.weight]
Loading weights:  40%|███▉      | 189/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.input_layernorm.weight]         
Loading weights:  40%|███▉      | 189/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.input_layernorm.weight]
Loading weights:  40%|████      | 190/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.A_log]     
Loading weights:  40%|████      | 190/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.A_log]
Loading weights:  40%|████      | 191/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.conv1d.weight]
Loading weights:  40%|████      | 191/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.conv1d.weight]
Loading weights:  41%|████      | 192/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.dt_bias]      
Loading weights:  41%|████      | 192/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.dt_bias]
Loading weights:  41%|████      | 193/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.in_proj_a.weight]
Loading weights:  41%|████      | 193/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.in_proj_a.weight]
Loading weights:  41%|████      | 194/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.in_proj_b.weight]
Loading weights:  41%|████      | 194/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.in_proj_b.weight]
Loading weights:  41%|████      | 195/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.in_proj_qkv.weight]
Loading weights:  41%|████      | 195/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.in_proj_qkv.weight]
Loading weights:  41%|████▏     | 196/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.in_proj_z.weight]  
Loading weights:  41%|████▏     | 196/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.in_proj_z.weight]
Loading weights:  42%|████▏     | 197/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.norm.weight]     
Loading weights:  42%|████▏     | 197/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.norm.weight]
Loading weights:  42%|████▏     | 198/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.out_proj.weight]
Loading weights:  42%|████▏     | 198/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.linear_attn.out_proj.weight]
Loading weights:  42%|████▏     | 199/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.mlp.down_proj.weight]       
Loading weights:  42%|████▏     | 199/473 [00:00<00:00, 321.81it/s, Materializing param=language_model.layers.14.mlp.down_proj.weight]
Loading weights:  42%|████▏     | 200/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.14.mlp.down_proj.weight]
Loading weights:  42%|████▏     | 200/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.14.mlp.gate_proj.weight]
Loading weights:  42%|████▏     | 200/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.14.mlp.gate_proj.weight]
Loading weights:  42%|████▏     | 201/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.14.mlp.up_proj.weight]  
Loading weights:  42%|████▏     | 201/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.14.mlp.up_proj.weight]
Loading weights:  43%|████▎     | 202/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.14.post_attention_layernorm.weight]
Loading weights:  43%|████▎     | 202/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.14.post_attention_layernorm.weight]
Loading weights:  43%|████▎     | 203/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.input_layernorm.weight]         
Loading weights:  43%|████▎     | 203/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.input_layernorm.weight]
Loading weights:  43%|████▎     | 204/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.mlp.down_proj.weight]  
Loading weights:  43%|████▎     | 204/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.mlp.down_proj.weight]
Loading weights:  43%|████▎     | 205/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.mlp.gate_proj.weight]
Loading weights:  43%|████▎     | 205/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.mlp.gate_proj.weight]
Loading weights:  44%|████▎     | 206/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.mlp.up_proj.weight]  
Loading weights:  44%|████▎     | 206/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.mlp.up_proj.weight]
Loading weights:  44%|████▍     | 207/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 207/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 208/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.self_attn.k_norm.weight]        
Loading weights:  44%|████▍     | 208/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.self_attn.k_norm.weight]
Loading weights:  44%|████▍     | 209/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 209/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 210/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.self_attn.o_proj.weight]
Loading weights:  44%|████▍     | 210/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.self_attn.o_proj.weight]
Loading weights:  45%|████▍     | 211/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 211/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 212/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.self_attn.q_proj.weight]
Loading weights:  45%|████▍     | 212/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.self_attn.q_proj.weight]
Loading weights:  45%|████▌     | 213/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 213/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.15.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 214/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.input_layernorm.weight] 
Loading weights:  45%|████▌     | 214/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.input_layernorm.weight]
Loading weights:  45%|████▌     | 215/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.A_log]     
Loading weights:  45%|████▌     | 215/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.A_log]
Loading weights:  46%|████▌     | 216/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.conv1d.weight]
Loading weights:  46%|████▌     | 216/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.conv1d.weight]
Loading weights:  46%|████▌     | 217/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.dt_bias]      
Loading weights:  46%|████▌     | 217/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.dt_bias]
Loading weights:  46%|████▌     | 218/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.in_proj_a.weight]
Loading weights:  46%|████▌     | 218/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.in_proj_a.weight]
Loading weights:  46%|████▋     | 219/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.in_proj_b.weight]
Loading weights:  46%|████▋     | 219/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.in_proj_b.weight]
Loading weights:  47%|████▋     | 220/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.in_proj_qkv.weight]
Loading weights:  47%|████▋     | 220/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.in_proj_qkv.weight]
Loading weights:  47%|████▋     | 221/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.in_proj_z.weight]  
Loading weights:  47%|████▋     | 221/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.in_proj_z.weight]
Loading weights:  47%|████▋     | 222/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.norm.weight]     
Loading weights:  47%|████▋     | 222/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.norm.weight]
Loading weights:  47%|████▋     | 223/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.out_proj.weight]
Loading weights:  47%|████▋     | 223/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.linear_attn.out_proj.weight]
Loading weights:  47%|████▋     | 224/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.mlp.down_proj.weight]       
Loading weights:  47%|████▋     | 224/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.mlp.down_proj.weight]
Loading weights:  48%|████▊     | 225/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.mlp.gate_proj.weight]
Loading weights:  48%|████▊     | 225/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.mlp.gate_proj.weight]
Loading weights:  48%|████▊     | 226/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.mlp.up_proj.weight]  
Loading weights:  48%|████▊     | 226/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.mlp.up_proj.weight]
Loading weights:  48%|████▊     | 227/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.post_attention_layernorm.weight]
Loading weights:  48%|████▊     | 227/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.16.post_attention_layernorm.weight]
Loading weights:  48%|████▊     | 228/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.input_layernorm.weight]         
Loading weights:  48%|████▊     | 228/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.input_layernorm.weight]
Loading weights:  48%|████▊     | 229/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.A_log]     
Loading weights:  48%|████▊     | 229/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.A_log]
Loading weights:  49%|████▊     | 230/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.conv1d.weight]
Loading weights:  49%|████▊     | 230/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.conv1d.weight]
Loading weights:  49%|████▉     | 231/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.dt_bias]      
Loading weights:  49%|████▉     | 231/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.dt_bias]
Loading weights:  49%|████▉     | 232/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.in_proj_a.weight]
Loading weights:  49%|████▉     | 232/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.in_proj_a.weight]
Loading weights:  49%|████▉     | 233/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.in_proj_b.weight]
Loading weights:  49%|████▉     | 233/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.in_proj_b.weight]
Loading weights:  49%|████▉     | 234/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.in_proj_qkv.weight]
Loading weights:  49%|████▉     | 234/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.in_proj_qkv.weight]
Loading weights:  50%|████▉     | 235/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.in_proj_z.weight]  
Loading weights:  50%|████▉     | 235/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.in_proj_z.weight]
Loading weights:  50%|████▉     | 236/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.norm.weight]     
Loading weights:  50%|████▉     | 236/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.norm.weight]
Loading weights:  50%|█████     | 237/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.out_proj.weight]
Loading weights:  50%|█████     | 237/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.linear_attn.out_proj.weight]
Loading weights:  50%|█████     | 238/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.mlp.down_proj.weight]       
Loading weights:  50%|█████     | 238/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.mlp.down_proj.weight]
Loading weights:  51%|█████     | 239/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 239/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 240/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.mlp.up_proj.weight]  
Loading weights:  51%|█████     | 240/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.mlp.up_proj.weight]
Loading weights:  51%|█████     | 241/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.post_attention_layernorm.weight]
Loading weights:  51%|█████     | 241/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.17.post_attention_layernorm.weight]
Loading weights:  51%|█████     | 242/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.input_layernorm.weight]         
Loading weights:  51%|█████     | 242/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.input_layernorm.weight]
Loading weights:  51%|█████▏    | 243/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.A_log]     
Loading weights:  51%|█████▏    | 243/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.A_log]
Loading weights:  52%|█████▏    | 244/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.conv1d.weight]
Loading weights:  52%|█████▏    | 244/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.conv1d.weight]
Loading weights:  52%|█████▏    | 245/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.dt_bias]      
Loading weights:  52%|█████▏    | 245/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.dt_bias]
Loading weights:  52%|█████▏    | 246/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.in_proj_a.weight]
Loading weights:  52%|█████▏    | 246/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.in_proj_a.weight]
Loading weights:  52%|█████▏    | 247/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.in_proj_b.weight]
Loading weights:  52%|█████▏    | 247/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.in_proj_b.weight]
Loading weights:  52%|█████▏    | 248/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.in_proj_qkv.weight]
Loading weights:  52%|█████▏    | 248/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.in_proj_qkv.weight]
Loading weights:  53%|█████▎    | 249/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.in_proj_z.weight]  
Loading weights:  53%|█████▎    | 249/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.in_proj_z.weight]
Loading weights:  53%|█████▎    | 250/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.norm.weight]     
Loading weights:  53%|█████▎    | 250/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.norm.weight]
Loading weights:  53%|█████▎    | 251/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.out_proj.weight]
Loading weights:  53%|█████▎    | 251/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.linear_attn.out_proj.weight]
Loading weights:  53%|█████▎    | 252/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.mlp.down_proj.weight]       
Loading weights:  53%|█████▎    | 252/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.mlp.down_proj.weight]
Loading weights:  53%|█████▎    | 253/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 253/473 [00:00<00:00, 359.68it/s, Materializing param=language_model.layers.18.mlp.gate_proj.weight]
Loading weights:  54%|█████▎    | 254/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.18.mlp.gate_proj.weight]
Loading weights:  54%|█████▎    | 254/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.18.mlp.up_proj.weight]  
Loading weights:  54%|█████▎    | 254/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.18.mlp.up_proj.weight]
Loading weights:  54%|█████▍    | 255/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.18.post_attention_layernorm.weight]
Loading weights:  54%|█████▍    | 255/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.18.post_attention_layernorm.weight]
Loading weights:  54%|█████▍    | 256/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.input_layernorm.weight]         
Loading weights:  54%|█████▍    | 256/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.input_layernorm.weight]
Loading weights:  54%|█████▍    | 257/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.mlp.down_proj.weight]  
Loading weights:  54%|█████▍    | 257/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.mlp.down_proj.weight]
Loading weights:  55%|█████▍    | 258/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.mlp.gate_proj.weight]
Loading weights:  55%|█████▍    | 258/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.mlp.gate_proj.weight]
Loading weights:  55%|█████▍    | 259/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.mlp.up_proj.weight]  
Loading weights:  55%|█████▍    | 259/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.mlp.up_proj.weight]
Loading weights:  55%|█████▍    | 260/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.post_attention_layernorm.weight]
Loading weights:  55%|█████▍    | 260/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.post_attention_layernorm.weight]
Loading weights:  55%|█████▌    | 261/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.self_attn.k_norm.weight]        
Loading weights:  55%|█████▌    | 261/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.self_attn.k_norm.weight]
Loading weights:  55%|█████▌    | 262/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.self_attn.k_proj.weight]
Loading weights:  55%|█████▌    | 262/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.self_attn.k_proj.weight]
Loading weights:  56%|█████▌    | 263/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.self_attn.o_proj.weight]
Loading weights:  56%|█████▌    | 263/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.self_attn.o_proj.weight]
Loading weights:  56%|█████▌    | 264/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.self_attn.q_norm.weight]
Loading weights:  56%|█████▌    | 264/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.self_attn.q_norm.weight]
Loading weights:  56%|█████▌    | 265/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.self_attn.q_proj.weight]
Loading weights:  56%|█████▌    | 265/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.self_attn.q_proj.weight]
Loading weights:  56%|█████▌    | 266/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.self_attn.v_proj.weight]
Loading weights:  56%|█████▌    | 266/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.19.self_attn.v_proj.weight]
Loading weights:  56%|█████▋    | 267/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.input_layernorm.weight] 
Loading weights:  56%|█████▋    | 267/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.input_layernorm.weight]
Loading weights:  57%|█████▋    | 268/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.A_log]     
Loading weights:  57%|█████▋    | 268/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.A_log]
Loading weights:  57%|█████▋    | 269/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.conv1d.weight]
Loading weights:  57%|█████▋    | 269/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.conv1d.weight]
Loading weights:  57%|█████▋    | 270/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.dt_bias]      
Loading weights:  57%|█████▋    | 270/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.dt_bias]
Loading weights:  57%|█████▋    | 271/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.in_proj_a.weight]
Loading weights:  57%|█████▋    | 271/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.in_proj_a.weight]
Loading weights:  58%|█████▊    | 272/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.in_proj_b.weight]
Loading weights:  58%|█████▊    | 272/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.in_proj_b.weight]
Loading weights:  58%|█████▊    | 273/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.in_proj_qkv.weight]
Loading weights:  58%|█████▊    | 273/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.in_proj_qkv.weight]
Loading weights:  58%|█████▊    | 274/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.in_proj_z.weight]  
Loading weights:  58%|█████▊    | 274/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.in_proj_z.weight]
Loading weights:  58%|█████▊    | 275/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.norm.weight]     
Loading weights:  58%|█████▊    | 275/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.norm.weight]
Loading weights:  58%|█████▊    | 276/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.out_proj.weight]
Loading weights:  58%|█████▊    | 276/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.linear_attn.out_proj.weight]
Loading weights:  59%|█████▊    | 277/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.mlp.down_proj.weight]       
Loading weights:  59%|█████▊    | 277/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.mlp.down_proj.weight]
Loading weights:  59%|█████▉    | 278/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.mlp.gate_proj.weight]
Loading weights:  59%|█████▉    | 278/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.mlp.gate_proj.weight]
Loading weights:  59%|█████▉    | 279/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.mlp.up_proj.weight]  
Loading weights:  59%|█████▉    | 279/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.mlp.up_proj.weight]
Loading weights:  59%|█████▉    | 280/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.post_attention_layernorm.weight]
Loading weights:  59%|█████▉    | 280/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.20.post_attention_layernorm.weight]
Loading weights:  59%|█████▉    | 281/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.input_layernorm.weight]         
Loading weights:  59%|█████▉    | 281/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.input_layernorm.weight]
Loading weights:  60%|█████▉    | 282/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.A_log]     
Loading weights:  60%|█████▉    | 282/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.A_log]
Loading weights:  60%|█████▉    | 283/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.conv1d.weight]
Loading weights:  60%|█████▉    | 283/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.conv1d.weight]
Loading weights:  60%|██████    | 284/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.dt_bias]      
Loading weights:  60%|██████    | 284/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.dt_bias]
Loading weights:  60%|██████    | 285/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.in_proj_a.weight]
Loading weights:  60%|██████    | 285/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.in_proj_a.weight]
Loading weights:  60%|██████    | 286/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.in_proj_b.weight]
Loading weights:  60%|██████    | 286/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.in_proj_b.weight]
Loading weights:  61%|██████    | 287/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.in_proj_qkv.weight]
Loading weights:  61%|██████    | 287/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.in_proj_qkv.weight]
Loading weights:  61%|██████    | 288/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.in_proj_z.weight]  
Loading weights:  61%|██████    | 288/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.in_proj_z.weight]
Loading weights:  61%|██████    | 289/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.norm.weight]     
Loading weights:  61%|██████    | 289/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.norm.weight]
Loading weights:  61%|██████▏   | 290/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.out_proj.weight]
Loading weights:  61%|██████▏   | 290/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.linear_attn.out_proj.weight]
Loading weights:  62%|██████▏   | 291/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.mlp.down_proj.weight]       
Loading weights:  62%|██████▏   | 291/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.mlp.down_proj.weight]
Loading weights:  62%|██████▏   | 292/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.mlp.gate_proj.weight]
Loading weights:  62%|██████▏   | 292/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.mlp.gate_proj.weight]
Loading weights:  62%|██████▏   | 293/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.mlp.up_proj.weight]  
Loading weights:  62%|██████▏   | 293/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.mlp.up_proj.weight]
Loading weights:  62%|██████▏   | 294/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.post_attention_layernorm.weight]
Loading weights:  62%|██████▏   | 294/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.21.post_attention_layernorm.weight]
Loading weights:  62%|██████▏   | 295/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.22.input_layernorm.weight]         
Loading weights:  62%|██████▏   | 295/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.22.input_layernorm.weight]
Loading weights:  63%|██████▎   | 296/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.22.linear_attn.A_log]     
Loading weights:  63%|██████▎   | 296/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.22.linear_attn.A_log]
Loading weights:  63%|██████▎   | 297/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.22.linear_attn.conv1d.weight]
Loading weights:  63%|██████▎   | 297/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.22.linear_attn.conv1d.weight]
Loading weights:  63%|██████▎   | 298/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.22.linear_attn.dt_bias]      
Loading weights:  63%|██████▎   | 298/473 [00:00<00:00, 408.43it/s, Materializing param=language_model.layers.22.linear_attn.dt_bias]
Loading weights:  63%|██████▎   | 299/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.linear_attn.dt_bias]
Loading weights:  63%|██████▎   | 299/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.linear_attn.in_proj_a.weight]
Loading weights:  63%|██████▎   | 299/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.linear_attn.in_proj_a.weight]
Loading weights:  63%|██████▎   | 300/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.linear_attn.in_proj_b.weight]
Loading weights:  63%|██████▎   | 300/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.linear_attn.in_proj_b.weight]
Loading weights:  64%|██████▎   | 301/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.linear_attn.in_proj_qkv.weight]
Loading weights:  64%|██████▎   | 301/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.linear_attn.in_proj_qkv.weight]
Loading weights:  64%|██████▍   | 302/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.linear_attn.in_proj_z.weight]  
Loading weights:  64%|██████▍   | 302/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.linear_attn.in_proj_z.weight]
Loading weights:  64%|██████▍   | 303/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.linear_attn.norm.weight]     
Loading weights:  64%|██████▍   | 303/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.linear_attn.norm.weight]
Loading weights:  64%|██████▍   | 304/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.linear_attn.out_proj.weight]
Loading weights:  64%|██████▍   | 304/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.linear_attn.out_proj.weight]
Loading weights:  64%|██████▍   | 305/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.mlp.down_proj.weight]       
Loading weights:  64%|██████▍   | 305/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.mlp.down_proj.weight]
Loading weights:  65%|██████▍   | 306/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.mlp.gate_proj.weight]
Loading weights:  65%|██████▍   | 306/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.mlp.gate_proj.weight]
Loading weights:  65%|██████▍   | 307/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.mlp.up_proj.weight]  
Loading weights:  65%|██████▍   | 307/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.mlp.up_proj.weight]
Loading weights:  65%|██████▌   | 308/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.post_attention_layernorm.weight]
Loading weights:  65%|██████▌   | 308/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.22.post_attention_layernorm.weight]
Loading weights:  65%|██████▌   | 309/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.input_layernorm.weight]         
Loading weights:  65%|██████▌   | 309/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.input_layernorm.weight]
Loading weights:  66%|██████▌   | 310/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.mlp.down_proj.weight]  
Loading weights:  66%|██████▌   | 310/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.mlp.down_proj.weight]
Loading weights:  66%|██████▌   | 311/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 311/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 312/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.mlp.up_proj.weight]  
Loading weights:  66%|██████▌   | 312/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.mlp.up_proj.weight]
Loading weights:  66%|██████▌   | 313/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.post_attention_layernorm.weight]
Loading weights:  66%|██████▌   | 313/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.post_attention_layernorm.weight]
Loading weights:  66%|██████▋   | 314/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.self_attn.k_norm.weight]        
Loading weights:  66%|██████▋   | 314/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.self_attn.k_norm.weight]
Loading weights:  67%|██████▋   | 315/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.self_attn.k_proj.weight]
Loading weights:  67%|██████▋   | 315/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.self_attn.k_proj.weight]
Loading weights:  67%|██████▋   | 316/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 316/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 317/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 317/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 318/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 318/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 319/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.self_attn.v_proj.weight]
Loading weights:  67%|██████▋   | 319/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.layers.23.self_attn.v_proj.weight]
Loading weights:  68%|██████▊   | 320/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.norm.weight]                      
Loading weights:  68%|██████▊   | 320/473 [00:00<00:00, 366.23it/s, Materializing param=language_model.norm.weight]
Loading weights:  68%|██████▊   | 321/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.attn.proj.bias]
Loading weights:  68%|██████▊   | 321/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.attn.proj.bias]
Loading weights:  68%|██████▊   | 322/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.attn.proj.weight]
Loading weights:  68%|██████▊   | 322/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.attn.proj.weight]
Loading weights:  68%|██████▊   | 323/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.attn.qkv.bias]   
Loading weights:  68%|██████▊   | 323/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.attn.qkv.bias]
Loading weights:  68%|██████▊   | 324/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.attn.qkv.weight]
Loading weights:  68%|██████▊   | 324/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.attn.qkv.weight]
Loading weights:  69%|██████▊   | 325/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.mlp.linear_fc1.bias]
Loading weights:  69%|██████▊   | 325/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.mlp.linear_fc1.bias]
Loading weights:  69%|██████▉   | 326/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.mlp.linear_fc1.weight]
Loading weights:  69%|██████▉   | 326/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.mlp.linear_fc1.weight]
Loading weights:  69%|██████▉   | 327/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.mlp.linear_fc2.bias]  
Loading weights:  69%|██████▉   | 327/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.mlp.linear_fc2.bias]
Loading weights:  69%|██████▉   | 328/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.mlp.linear_fc2.weight]
Loading weights:  69%|██████▉   | 328/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.mlp.linear_fc2.weight]
Loading weights:  70%|██████▉   | 329/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.norm1.bias]           
Loading weights:  70%|██████▉   | 329/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.norm1.bias]
Loading weights:  70%|██████▉   | 330/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.norm1.weight]
Loading weights:  70%|██████▉   | 330/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.norm1.weight]
Loading weights:  70%|██████▉   | 331/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.norm2.bias]  
Loading weights:  70%|██████▉   | 331/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.norm2.bias]
Loading weights:  70%|███████   | 332/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.norm2.weight]
Loading weights:  70%|███████   | 332/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.0.norm2.weight]
Loading weights:  70%|███████   | 333/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.1.attn.proj.bias]
Loading weights:  70%|███████   | 333/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.1.attn.proj.bias]
Loading weights:  71%|███████   | 334/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.1.attn.proj.weight]
Loading weights:  71%|███████   | 334/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.1.attn.proj.weight]
Loading weights:  71%|███████   | 335/473 [00:00<00:00, 366.23it/s, Materializing param=visual.blocks.1.attn.qkv.bias]   
Loading weights:  71%|███████   | 335/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.attn.qkv.bias]
Loading weights:  71%|███████   | 336/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.attn.qkv.weight]
Loading weights:  71%|███████   | 336/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.attn.qkv.weight]
Loading weights:  71%|███████   | 337/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.mlp.linear_fc1.bias]
Loading weights:  71%|███████   | 337/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.mlp.linear_fc1.bias]
Loading weights:  71%|███████▏  | 338/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.mlp.linear_fc1.weight]
Loading weights:  71%|███████▏  | 338/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.mlp.linear_fc1.weight]
Loading weights:  72%|███████▏  | 339/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.mlp.linear_fc2.bias]  
Loading weights:  72%|███████▏  | 339/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.mlp.linear_fc2.bias]
Loading weights:  72%|███████▏  | 340/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.mlp.linear_fc2.weight]
Loading weights:  72%|███████▏  | 340/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.mlp.linear_fc2.weight]
Loading weights:  72%|███████▏  | 341/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.norm1.bias]           
Loading weights:  72%|███████▏  | 341/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.norm1.bias]
Loading weights:  72%|███████▏  | 342/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.norm1.weight]
Loading weights:  72%|███████▏  | 342/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.norm1.weight]
Loading weights:  73%|███████▎  | 343/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.norm2.bias]  
Loading weights:  73%|███████▎  | 343/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.norm2.bias]
Loading weights:  73%|███████▎  | 344/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.norm2.weight]
Loading weights:  73%|███████▎  | 344/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.1.norm2.weight]
Loading weights:  73%|███████▎  | 345/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.attn.proj.bias]
Loading weights:  73%|███████▎  | 345/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.attn.proj.bias]
Loading weights:  73%|███████▎  | 346/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.attn.proj.weight]
Loading weights:  73%|███████▎  | 346/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.attn.proj.weight]
Loading weights:  73%|███████▎  | 347/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.attn.qkv.bias]   
Loading weights:  73%|███████▎  | 347/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.attn.qkv.bias]
Loading weights:  74%|███████▎  | 348/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.attn.qkv.weight]
Loading weights:  74%|███████▎  | 348/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.attn.qkv.weight]
Loading weights:  74%|███████▍  | 349/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.mlp.linear_fc1.bias]
Loading weights:  74%|███████▍  | 349/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.mlp.linear_fc1.bias]
Loading weights:  74%|███████▍  | 350/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.mlp.linear_fc1.weight]
Loading weights:  74%|███████▍  | 350/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.mlp.linear_fc1.weight]
Loading weights:  74%|███████▍  | 351/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.mlp.linear_fc2.bias]  
Loading weights:  74%|███████▍  | 351/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.mlp.linear_fc2.bias]
Loading weights:  74%|███████▍  | 352/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.mlp.linear_fc2.weight]
Loading weights:  74%|███████▍  | 352/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.mlp.linear_fc2.weight]
Loading weights:  75%|███████▍  | 353/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.norm1.bias]           
Loading weights:  75%|███████▍  | 353/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.norm1.bias]
Loading weights:  75%|███████▍  | 354/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.norm1.weight]
Loading weights:  75%|███████▍  | 354/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.norm1.weight]
Loading weights:  75%|███████▌  | 355/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.norm2.bias]  
Loading weights:  75%|███████▌  | 355/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.norm2.bias]
Loading weights:  75%|███████▌  | 356/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.norm2.weight]
Loading weights:  75%|███████▌  | 356/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.2.norm2.weight]
Loading weights:  75%|███████▌  | 357/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.attn.proj.bias]
Loading weights:  75%|███████▌  | 357/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.attn.proj.bias]
Loading weights:  76%|███████▌  | 358/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.attn.proj.weight]
Loading weights:  76%|███████▌  | 358/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.attn.proj.weight]
Loading weights:  76%|███████▌  | 359/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.attn.qkv.bias]   
Loading weights:  76%|███████▌  | 359/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.attn.qkv.bias]
Loading weights:  76%|███████▌  | 360/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.attn.qkv.weight]
Loading weights:  76%|███████▌  | 360/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.attn.qkv.weight]
Loading weights:  76%|███████▋  | 361/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.mlp.linear_fc1.bias]
Loading weights:  76%|███████▋  | 361/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.mlp.linear_fc1.bias]
Loading weights:  77%|███████▋  | 362/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.mlp.linear_fc1.weight]
Loading weights:  77%|███████▋  | 362/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.mlp.linear_fc1.weight]
Loading weights:  77%|███████▋  | 363/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.mlp.linear_fc2.bias]  
Loading weights:  77%|███████▋  | 363/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.mlp.linear_fc2.bias]
Loading weights:  77%|███████▋  | 364/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.mlp.linear_fc2.weight]
Loading weights:  77%|███████▋  | 364/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.mlp.linear_fc2.weight]
Loading weights:  77%|███████▋  | 365/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.norm1.bias]           
Loading weights:  77%|███████▋  | 365/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.norm1.bias]
Loading weights:  77%|███████▋  | 366/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.norm1.weight]
Loading weights:  77%|███████▋  | 366/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.norm1.weight]
Loading weights:  78%|███████▊  | 367/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.norm2.bias]  
Loading weights:  78%|███████▊  | 367/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.norm2.bias]
Loading weights:  78%|███████▊  | 368/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.norm2.weight]
Loading weights:  78%|███████▊  | 368/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.3.norm2.weight]
Loading weights:  78%|███████▊  | 369/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.attn.proj.bias]
Loading weights:  78%|███████▊  | 369/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.attn.proj.bias]
Loading weights:  78%|███████▊  | 370/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.attn.proj.weight]
Loading weights:  78%|███████▊  | 370/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.attn.proj.weight]
Loading weights:  78%|███████▊  | 371/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.attn.qkv.bias]   
Loading weights:  78%|███████▊  | 371/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.attn.qkv.bias]
Loading weights:  79%|███████▊  | 372/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.attn.qkv.weight]
Loading weights:  79%|███████▊  | 372/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.attn.qkv.weight]
Loading weights:  79%|███████▉  | 373/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.mlp.linear_fc1.bias]
Loading weights:  79%|███████▉  | 373/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.mlp.linear_fc1.bias]
Loading weights:  79%|███████▉  | 374/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.mlp.linear_fc1.weight]
Loading weights:  79%|███████▉  | 374/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.mlp.linear_fc1.weight]
Loading weights:  79%|███████▉  | 375/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.mlp.linear_fc2.bias]  
Loading weights:  79%|███████▉  | 375/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.mlp.linear_fc2.bias]
Loading weights:  79%|███████▉  | 376/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.mlp.linear_fc2.weight]
Loading weights:  79%|███████▉  | 376/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.mlp.linear_fc2.weight]
Loading weights:  80%|███████▉  | 377/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.norm1.bias]           
Loading weights:  80%|███████▉  | 377/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.norm1.bias]
Loading weights:  80%|███████▉  | 378/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.norm1.weight]
Loading weights:  80%|███████▉  | 378/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.norm1.weight]
Loading weights:  80%|████████  | 379/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.norm2.bias]  
Loading weights:  80%|████████  | 379/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.norm2.bias]
Loading weights:  80%|████████  | 380/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.norm2.weight]
Loading weights:  80%|████████  | 380/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.4.norm2.weight]
Loading weights:  81%|████████  | 381/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.attn.proj.bias]
Loading weights:  81%|████████  | 381/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.attn.proj.bias]
Loading weights:  81%|████████  | 382/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.attn.proj.weight]
Loading weights:  81%|████████  | 382/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.attn.proj.weight]
Loading weights:  81%|████████  | 383/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.attn.qkv.bias]   
Loading weights:  81%|████████  | 383/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.attn.qkv.bias]
Loading weights:  81%|████████  | 384/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.attn.qkv.weight]
Loading weights:  81%|████████  | 384/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.attn.qkv.weight]
Loading weights:  81%|████████▏ | 385/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.mlp.linear_fc1.bias]
Loading weights:  81%|████████▏ | 385/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.mlp.linear_fc1.bias]
Loading weights:  82%|████████▏ | 386/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.mlp.linear_fc1.weight]
Loading weights:  82%|████████▏ | 386/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.mlp.linear_fc1.weight]
Loading weights:  82%|████████▏ | 387/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.mlp.linear_fc2.bias]  
Loading weights:  82%|████████▏ | 387/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.mlp.linear_fc2.bias]
Loading weights:  82%|████████▏ | 388/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.mlp.linear_fc2.weight]
Loading weights:  82%|████████▏ | 388/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.mlp.linear_fc2.weight]
Loading weights:  82%|████████▏ | 389/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.norm1.bias]           
Loading weights:  82%|████████▏ | 389/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.norm1.bias]
Loading weights:  82%|████████▏ | 390/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.norm1.weight]
Loading weights:  82%|████████▏ | 390/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.norm1.weight]
Loading weights:  83%|████████▎ | 391/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.norm2.bias]  
Loading weights:  83%|████████▎ | 391/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.norm2.bias]
Loading weights:  83%|████████▎ | 392/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.norm2.weight]
Loading weights:  83%|████████▎ | 392/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.5.norm2.weight]
Loading weights:  83%|████████▎ | 393/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.attn.proj.bias]
Loading weights:  83%|████████▎ | 393/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.attn.proj.bias]
Loading weights:  83%|████████▎ | 394/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.attn.proj.weight]
Loading weights:  83%|████████▎ | 394/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.attn.proj.weight]
Loading weights:  84%|████████▎ | 395/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.attn.qkv.bias]   
Loading weights:  84%|████████▎ | 395/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.attn.qkv.bias]
Loading weights:  84%|████████▎ | 396/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.attn.qkv.weight]
Loading weights:  84%|████████▎ | 396/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.attn.qkv.weight]
Loading weights:  84%|████████▍ | 397/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.mlp.linear_fc1.bias]
Loading weights:  84%|████████▍ | 397/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.mlp.linear_fc1.bias]
Loading weights:  84%|████████▍ | 398/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.mlp.linear_fc1.weight]
Loading weights:  84%|████████▍ | 398/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.mlp.linear_fc1.weight]
Loading weights:  84%|████████▍ | 399/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.mlp.linear_fc2.bias]  
Loading weights:  84%|████████▍ | 399/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.mlp.linear_fc2.bias]
Loading weights:  85%|████████▍ | 400/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.mlp.linear_fc2.weight]
Loading weights:  85%|████████▍ | 400/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.mlp.linear_fc2.weight]
Loading weights:  85%|████████▍ | 401/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.norm1.bias]           
Loading weights:  85%|████████▍ | 401/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.norm1.bias]
Loading weights:  85%|████████▍ | 402/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.norm1.weight]
Loading weights:  85%|████████▍ | 402/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.norm1.weight]
Loading weights:  85%|████████▌ | 403/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.norm2.bias]  
Loading weights:  85%|████████▌ | 403/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.norm2.bias]
Loading weights:  85%|████████▌ | 404/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.norm2.weight]
Loading weights:  85%|████████▌ | 404/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.6.norm2.weight]
Loading weights:  86%|████████▌ | 405/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.attn.proj.bias]
Loading weights:  86%|████████▌ | 405/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.attn.proj.bias]
Loading weights:  86%|████████▌ | 406/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.attn.proj.weight]
Loading weights:  86%|████████▌ | 406/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.attn.proj.weight]
Loading weights:  86%|████████▌ | 407/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.attn.qkv.bias]   
Loading weights:  86%|████████▌ | 407/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.attn.qkv.bias]
Loading weights:  86%|████████▋ | 408/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.attn.qkv.weight]
Loading weights:  86%|████████▋ | 408/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.attn.qkv.weight]
Loading weights:  86%|████████▋ | 409/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.mlp.linear_fc1.bias]
Loading weights:  86%|████████▋ | 409/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.mlp.linear_fc1.bias]
Loading weights:  87%|████████▋ | 410/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.mlp.linear_fc1.weight]
Loading weights:  87%|████████▋ | 410/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.mlp.linear_fc1.weight]
Loading weights:  87%|████████▋ | 411/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.mlp.linear_fc2.bias]  
Loading weights:  87%|████████▋ | 411/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.mlp.linear_fc2.bias]
Loading weights:  87%|████████▋ | 412/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.mlp.linear_fc2.weight]
Loading weights:  87%|████████▋ | 412/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.mlp.linear_fc2.weight]
Loading weights:  87%|████████▋ | 413/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.norm1.bias]           
Loading weights:  87%|████████▋ | 413/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.norm1.bias]
Loading weights:  88%|████████▊ | 414/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.norm1.weight]
Loading weights:  88%|████████▊ | 414/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.norm1.weight]
Loading weights:  88%|████████▊ | 415/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.norm2.bias]  
Loading weights:  88%|████████▊ | 415/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.norm2.bias]
Loading weights:  88%|████████▊ | 416/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.norm2.weight]
Loading weights:  88%|████████▊ | 416/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.7.norm2.weight]
Loading weights:  88%|████████▊ | 417/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.8.attn.proj.bias]
Loading weights:  88%|████████▊ | 417/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.8.attn.proj.bias]
Loading weights:  88%|████████▊ | 418/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.8.attn.proj.weight]
Loading weights:  88%|████████▊ | 418/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.8.attn.proj.weight]
Loading weights:  89%|████████▊ | 419/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.8.attn.qkv.bias]   
Loading weights:  89%|████████▊ | 419/473 [00:01<00:00, 366.23it/s, Materializing param=visual.blocks.8.attn.qkv.bias]
Loading weights:  89%|████████▉ | 420/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.attn.qkv.bias]
Loading weights:  89%|████████▉ | 420/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.attn.qkv.weight]
Loading weights:  89%|████████▉ | 420/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.attn.qkv.weight]
Loading weights:  89%|████████▉ | 421/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.mlp.linear_fc1.bias]
Loading weights:  89%|████████▉ | 421/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.mlp.linear_fc1.bias]
Loading weights:  89%|████████▉ | 422/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.mlp.linear_fc1.weight]
Loading weights:  89%|████████▉ | 422/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.mlp.linear_fc1.weight]
Loading weights:  89%|████████▉ | 423/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.mlp.linear_fc2.bias]  
Loading weights:  89%|████████▉ | 423/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.mlp.linear_fc2.bias]
Loading weights:  90%|████████▉ | 424/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.mlp.linear_fc2.weight]
Loading weights:  90%|████████▉ | 424/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.mlp.linear_fc2.weight]
Loading weights:  90%|████████▉ | 425/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.norm1.bias]           
Loading weights:  90%|████████▉ | 425/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.norm1.bias]
Loading weights:  90%|█████████ | 426/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.norm1.weight]
Loading weights:  90%|█████████ | 426/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.norm1.weight]
Loading weights:  90%|█████████ | 427/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.norm2.bias]  
Loading weights:  90%|█████████ | 427/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.norm2.bias]
Loading weights:  90%|█████████ | 428/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.norm2.weight]
Loading weights:  90%|█████████ | 428/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.8.norm2.weight]
Loading weights:  91%|█████████ | 429/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.attn.proj.bias]
Loading weights:  91%|█████████ | 429/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.attn.proj.bias]
Loading weights:  91%|█████████ | 430/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.attn.proj.weight]
Loading weights:  91%|█████████ | 430/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.attn.proj.weight]
Loading weights:  91%|█████████ | 431/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.attn.qkv.bias]   
Loading weights:  91%|█████████ | 431/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.attn.qkv.bias]
Loading weights:  91%|█████████▏| 432/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.attn.qkv.weight]
Loading weights:  91%|█████████▏| 432/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.attn.qkv.weight]
Loading weights:  92%|█████████▏| 433/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.mlp.linear_fc1.bias]
Loading weights:  92%|█████████▏| 433/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.mlp.linear_fc1.bias]
Loading weights:  92%|█████████▏| 434/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.mlp.linear_fc1.weight]
Loading weights:  92%|█████████▏| 434/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.mlp.linear_fc1.weight]
Loading weights:  92%|█████████▏| 435/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.mlp.linear_fc2.bias]  
Loading weights:  92%|█████████▏| 435/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.mlp.linear_fc2.bias]
Loading weights:  92%|█████████▏| 436/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.mlp.linear_fc2.weight]
Loading weights:  92%|█████████▏| 436/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.mlp.linear_fc2.weight]
Loading weights:  92%|█████████▏| 437/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.norm1.bias]           
Loading weights:  92%|█████████▏| 437/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.norm1.bias]
Loading weights:  93%|█████████▎| 438/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.norm1.weight]
Loading weights:  93%|█████████▎| 438/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.norm1.weight]
Loading weights:  93%|█████████▎| 439/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.norm2.bias]  
Loading weights:  93%|█████████▎| 439/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.norm2.bias]
Loading weights:  93%|█████████▎| 440/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.norm2.weight]
Loading weights:  93%|█████████▎| 440/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.9.norm2.weight]
Loading weights:  93%|█████████▎| 441/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.attn.proj.bias]
Loading weights:  93%|█████████▎| 441/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.attn.proj.bias]
Loading weights:  93%|█████████▎| 442/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.attn.proj.weight]
Loading weights:  93%|█████████▎| 442/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.attn.proj.weight]
Loading weights:  94%|█████████▎| 443/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.attn.qkv.bias]   
Loading weights:  94%|█████████▎| 443/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.attn.qkv.bias]
Loading weights:  94%|█████████▍| 444/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.attn.qkv.weight]
Loading weights:  94%|█████████▍| 444/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.attn.qkv.weight]
Loading weights:  94%|█████████▍| 445/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.mlp.linear_fc1.bias]
Loading weights:  94%|█████████▍| 445/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.mlp.linear_fc1.bias]
Loading weights:  94%|█████████▍| 446/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.mlp.linear_fc1.weight]
Loading weights:  94%|█████████▍| 446/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.mlp.linear_fc1.weight]
Loading weights:  95%|█████████▍| 447/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.mlp.linear_fc2.bias]  
Loading weights:  95%|█████████▍| 447/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.mlp.linear_fc2.bias]
Loading weights:  95%|█████████▍| 448/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.mlp.linear_fc2.weight]
Loading weights:  95%|█████████▍| 448/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.mlp.linear_fc2.weight]
Loading weights:  95%|█████████▍| 449/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.norm1.bias]           
Loading weights:  95%|█████████▍| 449/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.norm1.bias]
Loading weights:  95%|█████████▌| 450/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.norm1.weight]
Loading weights:  95%|█████████▌| 450/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.norm1.weight]
Loading weights:  95%|█████████▌| 451/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.norm2.bias]  
Loading weights:  95%|█████████▌| 451/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.norm2.bias]
Loading weights:  96%|█████████▌| 452/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.norm2.weight]
Loading weights:  96%|█████████▌| 452/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.10.norm2.weight]
Loading weights:  96%|█████████▌| 453/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.attn.proj.bias]
Loading weights:  96%|█████████▌| 453/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.attn.proj.bias]
Loading weights:  96%|█████████▌| 454/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.attn.proj.weight]
Loading weights:  96%|█████████▌| 454/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.attn.proj.weight]
Loading weights:  96%|█████████▌| 455/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.attn.qkv.bias]   
Loading weights:  96%|█████████▌| 455/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.attn.qkv.bias]
Loading weights:  96%|█████████▋| 456/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.attn.qkv.weight]
Loading weights:  96%|█████████▋| 456/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.attn.qkv.weight]
Loading weights:  97%|█████████▋| 457/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.mlp.linear_fc1.bias]
Loading weights:  97%|█████████▋| 457/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.mlp.linear_fc1.bias]
Loading weights:  97%|█████████▋| 458/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.mlp.linear_fc1.weight]
Loading weights:  97%|█████████▋| 458/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.mlp.linear_fc1.weight]
Loading weights:  97%|█████████▋| 459/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.mlp.linear_fc2.bias]  
Loading weights:  97%|█████████▋| 459/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.mlp.linear_fc2.bias]
Loading weights:  97%|█████████▋| 460/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.mlp.linear_fc2.weight]
Loading weights:  97%|█████████▋| 460/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.mlp.linear_fc2.weight]
Loading weights:  97%|█████████▋| 461/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.norm1.bias]           
Loading weights:  97%|█████████▋| 461/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.norm1.bias]
Loading weights:  98%|█████████▊| 462/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.norm1.weight]
Loading weights:  98%|█████████▊| 462/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.norm1.weight]
Loading weights:  98%|█████████▊| 463/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.norm2.bias]  
Loading weights:  98%|█████████▊| 463/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.norm2.bias]
Loading weights:  98%|█████████▊| 464/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.norm2.weight]
Loading weights:  98%|█████████▊| 464/473 [00:01<00:00, 594.62it/s, Materializing param=visual.blocks.11.norm2.weight]
Loading weights:  98%|█████████▊| 465/473 [00:01<00:00, 594.62it/s, Materializing param=visual.merger.linear_fc1.bias]
Loading weights:  98%|█████████▊| 465/473 [00:01<00:00, 594.62it/s, Materializing param=visual.merger.linear_fc1.bias]
Loading weights:  99%|█████████▊| 466/473 [00:01<00:00, 594.62it/s, Materializing param=visual.merger.linear_fc1.weight]
Loading weights:  99%|█████████▊| 466/473 [00:01<00:00, 594.62it/s, Materializing param=visual.merger.linear_fc1.weight]
Loading weights:  99%|█████████▊| 467/473 [00:01<00:00, 594.62it/s, Materializing param=visual.merger.linear_fc2.bias]  
Loading weights:  99%|█████████▊| 467/473 [00:01<00:00, 594.62it/s, Materializing param=visual.merger.linear_fc2.bias]
Loading weights:  99%|█████████▉| 468/473 [00:01<00:00, 594.62it/s, Materializing param=visual.merger.linear_fc2.weight]
Loading weights:  99%|█████████▉| 468/473 [00:01<00:00, 594.62it/s, Materializing param=visual.merger.linear_fc2.weight]
Loading weights:  99%|█████████▉| 469/473 [00:01<00:00, 594.62it/s, Materializing param=visual.merger.norm.bias]        
Loading weights:  99%|█████████▉| 469/473 [00:01<00:00, 594.62it/s, Materializing param=visual.merger.norm.bias]
Loading weights:  99%|█████████▉| 470/473 [00:01<00:00, 594.62it/s, Materializing param=visual.merger.norm.weight]
Loading weights:  99%|█████████▉| 470/473 [00:01<00:00, 594.62it/s, Materializing param=visual.merger.norm.weight]
Loading weights: 100%|█████████▉| 471/473 [00:01<00:00, 594.62it/s, Materializing param=visual.patch_embed.proj.bias]
Loading weights: 100%|█████████▉| 471/473 [00:01<00:00, 594.62it/s, Materializing param=visual.patch_embed.proj.bias]
Loading weights: 100%|█████████▉| 472/473 [00:01<00:00, 594.62it/s, Materializing param=visual.patch_embed.proj.weight]
Loading weights: 100%|█████████▉| 472/473 [00:01<00:00, 594.62it/s, Materializing param=visual.patch_embed.proj.weight]
Loading weights: 100%|██████████| 473/473 [00:01<00:00, 594.62it/s, Materializing param=visual.pos_embed.weight]       
Loading weights: 100%|██████████| 473/473 [00:01<00:00, 594.62it/s, Materializing param=visual.pos_embed.weight]
Loading weights: 100%|██████████| 473/473 [00:01<00:00, 449.91it/s, Materializing param=visual.pos_embed.weight]
  Teacher loaded: 853.0M params
Loading student from MVP/artifacts/qwen35-0.8b-text-n4...

Loading weights:   0%|          | 0/55 [00:00<?, ?it/s]
Loading weights:   2%|▏         | 1/55 [00:00<00:00, 30393.51it/s, Materializing param=model.embed_tokens.weight]
Loading weights:   2%|▏         | 1/55 [00:00<00:00, 6563.86it/s, Materializing param=model.embed_tokens.weight] 
Loading weights:   4%|▎         | 2/55 [00:00<00:09,  5.60it/s, Materializing param=model.embed_tokens.weight]  
Loading weights:   4%|▎         | 2/55 [00:00<00:09,  5.60it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   4%|▎         | 2/55 [00:00<00:09,  5.60it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   5%|▌         | 3/55 [00:00<00:09,  5.60it/s, Materializing param=model.layers.0.linear_attn.A_log]     
Loading weights:   5%|▌         | 3/55 [00:00<00:09,  5.60it/s, Materializing param=model.layers.0.linear_attn.A_log]
Loading weights:   7%|▋         | 4/55 [00:00<00:09,  5.60it/s, Materializing param=model.layers.0.linear_attn.conv1d.weight]
Loading weights:   7%|▋         | 4/55 [00:00<00:09,  5.60it/s, Materializing param=model.layers.0.linear_attn.conv1d.weight]
Loading weights:   9%|▉         | 5/55 [00:00<00:08,  5.60it/s, Materializing param=model.layers.0.linear_attn.dt_bias]      
Loading weights:   9%|▉         | 5/55 [00:00<00:08,  5.60it/s, Materializing param=model.layers.0.linear_attn.dt_bias]
Loading weights:  11%|█         | 6/55 [00:00<00:08,  5.60it/s, Materializing param=model.layers.0.linear_attn.in_proj_a.weight]
Loading weights:  11%|█         | 6/55 [00:00<00:08,  5.60it/s, Materializing param=model.layers.0.linear_attn.in_proj_a.weight]
Loading weights:  13%|█▎        | 7/55 [00:00<00:08,  5.60it/s, Materializing param=model.layers.0.linear_attn.in_proj_b.weight]
Loading weights:  13%|█▎        | 7/55 [00:00<00:08,  5.60it/s, Materializing param=model.layers.0.linear_attn.in_proj_b.weight]
Loading weights:  15%|█▍        | 8/55 [00:00<00:08,  5.60it/s, Materializing param=model.layers.0.linear_attn.in_proj_qkv.weight]
Loading weights:  15%|█▍        | 8/55 [00:00<00:08,  5.60it/s, Materializing param=model.layers.0.linear_attn.in_proj_qkv.weight]
Loading weights:  16%|█▋        | 9/55 [00:00<00:08,  5.60it/s, Materializing param=model.layers.0.linear_attn.in_proj_z.weight]  
Loading weights:  16%|█▋        | 9/55 [00:00<00:08,  5.60it/s, Materializing param=model.layers.0.linear_attn.in_proj_z.weight]
Loading weights:  18%|█▊        | 10/55 [00:00<00:08,  5.60it/s, Materializing param=model.layers.0.linear_attn.norm.weight]    
Loading weights:  18%|█▊        | 10/55 [00:00<00:08,  5.60it/s, Materializing param=model.layers.0.linear_attn.norm.weight]
Loading weights:  20%|██        | 11/55 [00:00<00:07,  5.60it/s, Materializing param=model.layers.0.linear_attn.out_proj.weight]
Loading weights:  20%|██        | 11/55 [00:00<00:07,  5.60it/s, Materializing param=model.layers.0.linear_attn.out_proj.weight]
Loading weights:  22%|██▏       | 12/55 [00:00<00:07,  5.60it/s, Materializing param=model.layers.0.mlp.down_proj.weight]       
Loading weights:  22%|██▏       | 12/55 [00:00<00:07,  5.60it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:  24%|██▎       | 13/55 [00:00<00:07,  5.60it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:  24%|██▎       | 13/55 [00:00<00:07,  5.60it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:  25%|██▌       | 14/55 [00:00<00:07,  5.60it/s, Materializing param=model.layers.0.mlp.up_proj.weight]  
Loading weights:  25%|██▌       | 14/55 [00:00<00:07,  5.60it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:  27%|██▋       | 15/55 [00:00<00:07,  5.60it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:  27%|██▋       | 15/55 [00:00<00:07,  5.60it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 16/55 [00:00<00:06,  5.60it/s, Materializing param=model.layers.1.input_layernorm.weight]         
Loading weights:  29%|██▉       | 16/55 [00:00<00:06,  5.60it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:  31%|███       | 17/55 [00:00<00:06,  5.60it/s, Materializing param=model.layers.1.linear_attn.A_log]     
Loading weights:  31%|███       | 17/55 [00:00<00:06,  5.60it/s, Materializing param=model.layers.1.linear_attn.A_log]
Loading weights:  33%|███▎      | 18/55 [00:00<00:06,  5.60it/s, Materializing param=model.layers.1.linear_attn.conv1d.weight]
Loading weights:  33%|███▎      | 18/55 [00:00<00:06,  5.60it/s, Materializing param=model.layers.1.linear_attn.conv1d.weight]
Loading weights:  35%|███▍      | 19/55 [00:00<00:06,  5.60it/s, Materializing param=model.layers.1.linear_attn.dt_bias]      
Loading weights:  35%|███▍      | 19/55 [00:00<00:06,  5.60it/s, Materializing param=model.layers.1.linear_attn.dt_bias]
Loading weights:  36%|███▋      | 20/55 [00:00<00:06,  5.60it/s, Materializing param=model.layers.1.linear_attn.in_proj_a.weight]
Loading weights:  36%|███▋      | 20/55 [00:00<00:06,  5.60it/s, Materializing param=model.layers.1.linear_attn.in_proj_a.weight]
Loading weights:  38%|███▊      | 21/55 [00:00<00:06,  5.60it/s, Materializing param=model.layers.1.linear_attn.in_proj_b.weight]
Loading weights:  38%|███▊      | 21/55 [00:00<00:06,  5.60it/s, Materializing param=model.layers.1.linear_attn.in_proj_b.weight]
Loading weights:  40%|████      | 22/55 [00:00<00:05,  5.60it/s, Materializing param=model.layers.1.linear_attn.in_proj_qkv.weight]
Loading weights:  40%|████      | 22/55 [00:00<00:05,  5.60it/s, Materializing param=model.layers.1.linear_attn.in_proj_qkv.weight]
Loading weights:  42%|████▏     | 23/55 [00:00<00:05,  5.60it/s, Materializing param=model.layers.1.linear_attn.in_proj_z.weight]  
Loading weights:  42%|████▏     | 23/55 [00:00<00:05,  5.60it/s, Materializing param=model.layers.1.linear_attn.in_proj_z.weight]
Loading weights:  44%|████▎     | 24/55 [00:00<00:05,  5.60it/s, Materializing param=model.layers.1.linear_attn.norm.weight]     
Loading weights:  44%|████▎     | 24/55 [00:00<00:05,  5.60it/s, Materializing param=model.layers.1.linear_attn.norm.weight]
Loading weights:  45%|████▌     | 25/55 [00:00<00:05,  5.60it/s, Materializing param=model.layers.1.linear_attn.out_proj.weight]
Loading weights:  45%|████▌     | 25/55 [00:00<00:05,  5.60it/s, Materializing param=model.layers.1.linear_attn.out_proj.weight]
Loading weights:  47%|████▋     | 26/55 [00:00<00:05,  5.60it/s, Materializing param=model.layers.1.mlp.down_proj.weight]       
Loading weights:  47%|████▋     | 26/55 [00:00<00:05,  5.60it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:  49%|████▉     | 27/55 [00:00<00:04,  5.60it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:  49%|████▉     | 27/55 [00:00<00:04,  5.60it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 28/55 [00:00<00:04,  5.60it/s, Materializing param=model.layers.1.mlp.up_proj.weight]  
Loading weights:  51%|█████     | 28/55 [00:00<00:04,  5.60it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:  53%|█████▎    | 29/55 [00:00<00:04,  5.60it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:  53%|█████▎    | 29/55 [00:00<00:04,  5.60it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:  55%|█████▍    | 30/55 [00:00<00:04,  5.60it/s, Materializing param=model.layers.2.input_layernorm.weight]         
Loading weights:  55%|█████▍    | 30/55 [00:00<00:04,  5.60it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:  56%|█████▋    | 31/55 [00:00<00:04,  5.60it/s, Materializing param=model.layers.2.linear_attn.A_log]     
Loading weights:  56%|█████▋    | 31/55 [00:00<00:04,  5.60it/s, Materializing param=model.layers.2.linear_attn.A_log]
Loading weights:  58%|█████▊    | 32/55 [00:00<00:04,  5.60it/s, Materializing param=model.layers.2.linear_attn.conv1d.weight]
Loading weights:  58%|█████▊    | 32/55 [00:00<00:04,  5.60it/s, Materializing param=model.layers.2.linear_attn.conv1d.weight]
Loading weights:  60%|██████    | 33/55 [00:00<00:03,  5.60it/s, Materializing param=model.layers.2.linear_attn.dt_bias]      
Loading weights:  60%|██████    | 33/55 [00:00<00:03,  5.60it/s, Materializing param=model.layers.2.linear_attn.dt_bias]
Loading weights:  62%|██████▏   | 34/55 [00:00<00:03,  5.60it/s, Materializing param=model.layers.2.linear_attn.in_proj_a.weight]
Loading weights:  62%|██████▏   | 34/55 [00:00<00:03,  5.60it/s, Materializing param=model.layers.2.linear_attn.in_proj_a.weight]
Loading weights:  64%|██████▎   | 35/55 [00:00<00:03,  5.60it/s, Materializing param=model.layers.2.linear_attn.in_proj_b.weight]
Loading weights:  64%|██████▎   | 35/55 [00:00<00:03,  5.60it/s, Materializing param=model.layers.2.linear_attn.in_proj_b.weight]
Loading weights:  65%|██████▌   | 36/55 [00:00<00:03,  5.60it/s, Materializing param=model.layers.2.linear_attn.in_proj_qkv.weight]
Loading weights:  65%|██████▌   | 36/55 [00:00<00:03,  5.60it/s, Materializing param=model.layers.2.linear_attn.in_proj_qkv.weight]
Loading weights:  67%|██████▋   | 37/55 [00:00<00:03,  5.60it/s, Materializing param=model.layers.2.linear_attn.in_proj_z.weight]  
Loading weights:  67%|██████▋   | 37/55 [00:00<00:03,  5.60it/s, Materializing param=model.layers.2.linear_attn.in_proj_z.weight]
Loading weights:  69%|██████▉   | 38/55 [00:00<00:03,  5.60it/s, Materializing param=model.layers.2.linear_attn.norm.weight]     
Loading weights:  69%|██████▉   | 38/55 [00:00<00:03,  5.60it/s, Materializing param=model.layers.2.linear_attn.norm.weight]
Loading weights:  71%|███████   | 39/55 [00:00<00:02,  5.60it/s, Materializing param=model.layers.2.linear_attn.out_proj.weight]
Loading weights:  71%|███████   | 39/55 [00:00<00:02,  5.60it/s, Materializing param=model.layers.2.linear_attn.out_proj.weight]
Loading weights:  73%|███████▎  | 40/55 [00:00<00:02,  5.60it/s, Materializing param=model.layers.2.mlp.down_proj.weight]       
Loading weights:  73%|███████▎  | 40/55 [00:00<00:02,  5.60it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:  75%|███████▍  | 41/55 [00:00<00:02,  5.60it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:  75%|███████▍  | 41/55 [00:00<00:02,  5.60it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:  76%|███████▋  | 42/55 [00:00<00:02,  5.60it/s, Materializing param=model.layers.2.mlp.up_proj.weight]  
Loading weights:  76%|███████▋  | 42/55 [00:00<00:02,  5.60it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:  78%|███████▊  | 43/55 [00:00<00:02,  5.60it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:  78%|███████▊  | 43/55 [00:00<00:02,  5.60it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:  80%|████████  | 44/55 [00:00<00:01,  5.60it/s, Materializing param=model.layers.3.input_layernorm.weight]         
Loading weights:  80%|████████  | 44/55 [00:00<00:01,  5.60it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:  82%|████████▏ | 45/55 [00:00<00:01,  5.60it/s, Materializing param=model.layers.3.mlp.down_proj.weight]  
Loading weights:  82%|████████▏ | 45/55 [00:00<00:01,  5.60it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:  84%|████████▎ | 46/55 [00:00<00:01,  5.60it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:  84%|████████▎ | 46/55 [00:00<00:01,  5.60it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:  85%|████████▌ | 47/55 [00:00<00:01,  5.60it/s, Materializing param=model.layers.3.mlp.up_proj.weight]  
Loading weights:  85%|████████▌ | 47/55 [00:00<00:01,  5.60it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:  87%|████████▋ | 48/55 [00:00<00:01,  5.60it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:  87%|████████▋ | 48/55 [00:00<00:01,  5.60it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:  89%|████████▉ | 49/55 [00:00<00:01,  5.60it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]        
Loading weights:  89%|████████▉ | 49/55 [00:00<00:01,  5.60it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights:  91%|█████████ | 50/55 [00:00<00:00,  5.60it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  91%|█████████ | 50/55 [00:00<00:00,  5.60it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  93%|█████████▎| 51/55 [00:00<00:00,  5.60it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  93%|█████████▎| 51/55 [00:00<00:00,  5.60it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  95%|█████████▍| 52/55 [00:00<00:00,  5.60it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  95%|█████████▍| 52/55 [00:00<00:00,  5.60it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  96%|█████████▋| 53/55 [00:00<00:00,  5.60it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  96%|█████████▋| 53/55 [00:00<00:00,  5.60it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  98%|█████████▊| 54/55 [00:00<00:00,  5.60it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  98%|█████████▊| 54/55 [00:00<00:00,  5.60it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights: 100%|██████████| 55/55 [00:00<00:00,  5.60it/s, Materializing param=model.norm.weight]                     
Loading weights: 100%|██████████| 55/55 [00:00<00:00,  5.60it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 55/55 [00:00<00:00, 149.22it/s, Materializing param=model.norm.weight]
The tokenizer you are loading from 'MVP/artifacts/qwen35-0.8b-text-n4' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
Token indices sequence length is longer than the specified maximum sequence length for this model (2505798 >  Student loaded: 337.3M params
Loading tokenizer...
Tokenizing dataset (/tmp/wik/usr/lib/python3.14/multiprocessing/resource_tracker.py:475: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown: {'/mp-z_hczt5i'}
  warnings.warn(
en=128
  Step 25/2000 | total=288.4217 ce=13.6517 mse=27.477000 | lr=1.50e-05 | 107s elapsed, ~8492s remaining
  Step 50/2000 | total=249.2028 ce=12.1995 mse=23.700326 | lr=3.00e-05 | 214s elapsed, ~8336s remaining
  Step 75/2000 | total=182.6658 ce=9.3742 mse=17.329160 | lr=4.50e-05 | 320s elapsed, ~8217s remaining
  Step 100/2000 | total=138.5065 ce=7.9229 mse=13.058353 | lr=6.00e-05 | 427s elapsed, ~8108s remaining
  Step 125/2000 | total=115.7030 ce=7.5666 mse=10.813636 | lr=7.50e-05 | 532s elapsed, ~7977s remaining
  Step 150/2000 | total=106.5914 ce=7.3182 mse=9.927319 | lr=9.00e-05 | 638s elapsed, ~7865s remaining
  Step 175/2000 | total=100.8314 ce=6.9378 mse=9.389356 | lr=1.05e-04 | 743s elapsed, ~7747s remaining
  Step 200/2000 | total=98.4073 ce=6.5909 mse=9.181641 | lr=1.25e-04 | 850s elapsed, ~7648s remaining
  Step 225/2000 | total=93.9046 ce=6.3370 mse=8.756758 | lr=1.40e-04 | 955s elapsed, ~7536s remaining
  Step 250/2000 | total=88.4071 ce=6.0484 mse=8.235866 | lr=1.55e-04 | 1539s elapsed, ~10774s remaining
Traceback (most recent call last):
  File "/home/void0x14/Documents/echo/MVP/distill.py", line 289, in <module>
    _main()
    ~~~~~^^
  File "/home/void0x14/Documents/echo/MVP/distill.py", line 285, in _main
    run_distill(DistillConfig(**{k.replace("-", "_"): v for k, v in vars(args).items()}))
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/void0x14/Documents/echo/MVP/distill.py", line 256, in run_distill
    save_checkpoint(student, student_config, args.output_dir, step + 1)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/void0x14/Documents/echo/MVP/distill.py", line 160, in save_checkpoint
    config_path.write_text(json.dumps(student_config.to_dict(), indent=2, sort_keys=True) + "\n", encoding="utf-8")
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.14/json/__init__.py", line 242, in dumps
    **kw).encode(obj)
          ~~~~~~^^^^^
  File "/usr/lib/python3.14/json/encoder.py", line 202, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.14/json/encoder.py", line 263, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.14/json/encoder.py", line 182, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
                    f'is not JSON serializable')
TypeError: Object of type Qwen3_5TextConfig is not JSON serializable