File size: 91,266 Bytes
027ce51 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 |
ๅ ่ฝฝ่ฎญ็ป้: /tmp/home/wzh/file/train_data.csv
ๅ ่ฝฝ้ช่ฏ้: /tmp/home/wzh/file/val_data.csv
ๆญฃๅจ่ฎก็ฎ็ฑปๅซๆ้...
่ฎก็ฎๅบ็็ฑปๅซๆ้: [0.58033253 3.61206429]
ๆญฃๅจไธ่ฝฝ/ๅ ่ฝฝๆจกๅ: roberta-base ...
Map: 0%| | 0/40000 [00:00<?, ? examples/s]
Map: 2%|โ | 1000/40000 [00:00<00:09, 4176.70 examples/s]
Map: 5%|โ | 2000/40000 [00:00<00:09, 4220.60 examples/s]
Map: 8%|โ | 3000/40000 [00:00<00:08, 4261.20 examples/s]
Map: 10%|โ | 4000/40000 [00:01<00:11, 3037.93 examples/s]
Map: 12%|โโ | 5000/40000 [00:01<00:10, 3398.31 examples/s]
Map: 15%|โโ | 6000/40000 [00:01<00:09, 3576.90 examples/s]
Map: 18%|โโ | 7000/40000 [00:01<00:08, 3772.83 examples/s]
Map: 20%|โโ | 8000/40000 [00:02<00:08, 3972.11 examples/s]
Map: 22%|โโโ | 9000/40000 [00:02<00:07, 4082.89 examples/s]
Map: 25%|โโโ | 10000/40000 [00:02<00:07, 4142.79 examples/s]
Map: 28%|โโโ | 11000/40000 [00:02<00:06, 4148.22 examples/s]
Map: 30%|โโโ | 12000/40000 [00:03<00:06, 4157.05 examples/s]
Map: 32%|โโโโ | 13000/40000 [00:03<00:06, 4189.95 examples/s]
Map: 35%|โโโโ | 14000/40000 [00:03<00:06, 4262.87 examples/s]
Map: 38%|โโโโ | 15000/40000 [00:03<00:05, 4302.67 examples/s]
Map: 40%|โโโโ | 16000/40000 [00:03<00:05, 4228.55 examples/s]
Map: 42%|โโโโโ | 17000/40000 [00:04<00:05, 4296.47 examples/s]
Map: 45%|โโโโโ | 18000/40000 [00:04<00:05, 4311.63 examples/s]
Map: 48%|โโโโโ | 19000/40000 [00:04<00:04, 4300.71 examples/s]
Map: 50%|โโโโโ | 20000/40000 [00:04<00:04, 4364.00 examples/s]
Map: 52%|โโโโโโ | 21000/40000 [00:05<00:04, 4197.62 examples/s]
Map: 55%|โโโโโโ | 22000/40000 [00:05<00:04, 4228.29 examples/s]
Map: 57%|โโโโโโ | 23000/40000 [00:05<00:03, 4266.51 examples/s]
Map: 60%|โโโโโโ | 24000/40000 [00:05<00:03, 4299.29 examples/s]
Map: 62%|โโโโโโโ | 25000/40000 [00:06<00:03, 4308.04 examples/s]
Map: 65%|โโโโโโโ | 26000/40000 [00:06<00:03, 4345.02 examples/s]
Map: 68%|โโโโโโโ | 27000/40000 [00:06<00:03, 4317.96 examples/s]
Map: 70%|โโโโโโโ | 28000/40000 [00:06<00:02, 4319.04 examples/s]
Map: 72%|โโโโโโโโ | 29000/40000 [00:07<00:02, 4343.22 examples/s]
Map: 75%|โโโโโโโโ | 30000/40000 [00:07<00:02, 4344.63 examples/s]
Map: 78%|โโโโโโโโ | 31000/40000 [00:07<00:02, 4362.97 examples/s]
Map: 80%|โโโโโโโโ | 32000/40000 [00:07<00:01, 4397.59 examples/s]
Map: 82%|โโโโโโโโโ | 33000/40000 [00:07<00:01, 4392.02 examples/s]
Map: 85%|โโโโโโโโโ | 34000/40000 [00:08<00:01, 3290.20 examples/s]
Map: 88%|โโโโโโโโโ | 35000/40000 [00:08<00:01, 3566.64 examples/s]
Map: 90%|โโโโโโโโโ | 36000/40000 [00:08<00:01, 3749.19 examples/s]
Map: 92%|โโโโโโโโโโ| 37000/40000 [00:09<00:00, 3928.32 examples/s]
Map: 95%|โโโโโโโโโโ| 38000/40000 [00:09<00:00, 4065.60 examples/s]
Map: 98%|โโโโโโโโโโ| 39000/40000 [00:09<00:00, 4125.08 examples/s]
Map: 100%|โโโโโโโโโโ| 40000/40000 [00:09<00:00, 4199.39 examples/s]
Map: 100%|โโโโโโโโโโ| 40000/40000 [00:09<00:00, 4089.96 examples/s]
Map: 0%| | 0/5000 [00:00<?, ? examples/s]
Map: 20%|โโ | 1000/5000 [00:00<00:00, 4687.37 examples/s]
Map: 40%|โโโโ | 2000/5000 [00:00<00:00, 4537.19 examples/s]
Map: 60%|โโโโโโ | 3000/5000 [00:00<00:00, 4452.60 examples/s]
Map: 80%|โโโโโโโโ | 4000/5000 [00:00<00:00, 4445.04 examples/s]
Map: 100%|โโโโโโโโโโ| 5000/5000 [00:01<00:00, 4391.97 examples/s]
Map: 100%|โโโโโโโโโโ| 5000/5000 [00:01<00:00, 4418.94 examples/s]
/tmp/home/wzh/final/train.py:145: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `CustomTrainer.__init__`. Use `processing_class` instead.
trainer = CustomTrainer(
Some weights of RobertaForSequenceClassification were not initialized from the model checkpoint at roberta-base and are newly initialized: ['classifier.dense.bias', 'classifier.dense.weight', 'classifier.out_proj.bias', 'classifier.out_proj.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
[I 2025-11-26 12:05:51,188] A new study created in memory with name: no-name-0b166e0d-c37b-4d46-a204-3b08905480a6
Some weights of RobertaForSequenceClassification were not initialized from the model checkpoint at roberta-base and are newly initialized: ['classifier.dense.bias', 'classifier.dense.weight', 'classifier.out_proj.bias', 'classifier.out_proj.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
==================================================
๐ ๅผๅง่ชๅจ่ถ
ๅๆฐๆ็ดข (็ฎๆ : ๆๅคงๅ Macro-F1)...
==================================================
0%| | 0/1565 [00:00<?, ?it/s]7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360481 [0] NCCL INFO cudaDriverVersion 12020
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360481 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth0
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360481 [0] NCCL INFO Bootstrap: Using eth0:10.0.0.13<0>
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360481 [0] NCCL INFO NCCL version 2.27.5+cuda12.9
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO NET/Plugin: Plugin name set by env to libnccl-net.so
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v10 symbol.
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v9 symbol.
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v8 symbol.
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v7 symbol.
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin v6 (v6)
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v10 symbol.
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v9 symbol.
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v8 symbol.
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v7 symbol.
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO Successfully loaded external plugin libnccl-net.so
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO Plugin Path : /opt/nccl-rdma-sharp-plugins/lib/libnccl-net.so
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO P2P plugin IBext
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth0
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO NET/IB : Using [0]mlx5_ib0:1/IB [RO]; OOB eth0:10.0.0.13<0>
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO Initialized NET plugin NCCL RDMA Plugin v6
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO Assigned NET plugin NCCL RDMA Plugin v6 to comm
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO Using network NCCL RDMA Plugin v6
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO Assigned NET plugin NCCL RDMA Plugin v6 to comm
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO Using network NCCL RDMA Plugin v6
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360879 [3] NCCL INFO Assigned NET plugin NCCL RDMA Plugin v6 to comm
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360879 [3] NCCL INFO Using network NCCL RDMA Plugin v6
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO Assigned NET plugin NCCL RDMA Plugin v6 to comm
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO Using network NCCL RDMA Plugin v6
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360879 [3] NCCL INFO ncclCommInitAll comm 0x12465150 rank 3 nranks 4 cudaDev 3 nvmlDev 7 busId 800000 commId 0xa5c975f20549b316 - Init START
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO ncclCommInitAll comm 0x12132bb0 rank 0 nranks 4 cudaDev 0 nvmlDev 4 busId 500000 commId 0xa5c975f20549b316 - Init START
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO ncclCommInitAll comm 0x12353800 rank 2 nranks 4 cudaDev 2 nvmlDev 6 busId 700000 commId 0xa5c975f20549b316 - Init START
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO ncclCommInitAll comm 0x12242130 rank 1 nranks 4 cudaDev 1 nvmlDev 5 busId 600000 commId 0xa5c975f20549b316 - Init START
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360879 [3] NCCL INFO RAS client listening socket at 127.0.0.1<28028>
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO Bootstrap timings total 0.000982 (create 0.000036, send 0.000108, recv 0.000413, ring 0.000085, delay 0.000000)
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO Bootstrap timings total 0.000995 (create 0.000028, send 0.000098, recv 0.000241, ring 0.000073, delay 0.000000)
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360879 [3] NCCL INFO Bootstrap timings total 0.001021 (create 0.000043, send 0.000119, recv 0.000335, ring 0.000103, delay 0.000001)
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO Bootstrap timings total 0.000960 (create 0.000032, send 0.000097, recv 0.000472, ring 0.000064, delay 0.000000)
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO Setting affinity for GPU 5 to 20-39
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO NVLS multicast support is not available on dev 1 (NVLS_NCHANNELS 0)
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO Setting affinity for GPU 6 to 20-39
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO NVLS multicast support is not available on dev 2 (NVLS_NCHANNELS 0)
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360879 [3] NCCL INFO Setting affinity for GPU 7 to 20-39
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360879 [3] NCCL INFO NVLS multicast support is not available on dev 3 (NVLS_NCHANNELS 0)
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO Setting affinity for GPU 4 to 20-39
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO NVLS multicast support is not available on dev 0 (NVLS_NCHANNELS 0)
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO comm 0x12353800 rank 2 nRanks 4 nNodes 1 localRanks 4 localRank 2 MNNVL 0
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO comm 0x12132bb0 rank 0 nRanks 4 nNodes 1 localRanks 4 localRank 0 MNNVL 0
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO comm 0x12242130 rank 1 nRanks 4 nNodes 1 localRanks 4 localRank 1 MNNVL 0
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 1/-1/-1->2->3 [2] 1/-1/-1->2->3 [3] 3/-1/-1->2->0 [4] 3/-1/-1->2->1 [5] 1/-1/-1->2->3 [6] 1/-1/-1->2->3 [7] 3/-1/-1->2->0
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360879 [3] NCCL INFO comm 0x12465150 rank 3 nRanks 4 nNodes 1 localRanks 4 localRank 3 MNNVL 0
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] -1/-1/-1->1->2 [2] -1/-1/-1->1->2 [3] -1/-1/-1->1->3 [4] 2/-1/-1->1->0 [5] -1/-1/-1->1->2 [6] -1/-1/-1->1->2 [7] -1/-1/-1->1->3
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO P2P Chunksize set to 524288
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO P2P Chunksize set to 524288
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360879 [3] NCCL INFO Trees [0] -1/-1/-1->3->2 [1] 2/-1/-1->3->0 [2] 2/-1/-1->3->0 [3] 1/-1/-1->3->2 [4] -1/-1/-1->3->2 [5] 2/-1/-1->3->0 [6] 2/-1/-1->3->0 [7] 1/-1/-1->3->2
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360879 [3] NCCL INFO P2P Chunksize set to 524288
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO Channel 00/08 : 0 1 2 3
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO Channel 01/08 : 0 3 2 1
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO Channel 02/08 : 0 3 1 2
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO Channel 03/08 : 0 2 1 3
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO Channel 04/08 : 0 1 2 3
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO Channel 05/08 : 0 3 2 1
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO Channel 06/08 : 0 3 1 2
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO Channel 07/08 : 0 2 1 3
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 3/-1/-1->0->-1 [2] 3/-1/-1->0->-1 [3] 2/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 3/-1/-1->0->-1 [6] 3/-1/-1->0->-1 [7] 2/-1/-1->0->-1
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO P2P Chunksize set to 524288
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so.
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360888 [1] NCCL INFO [Proxy Service] Device 1 CPU core 22
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360889 [1] NCCL INFO [Proxy Service UDS] Device 1 CPU core 23
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so.
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360890 [2] NCCL INFO [Proxy Service] Device 2 CPU core 28
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360891 [2] NCCL INFO [Proxy Service UDS] Device 2 CPU core 30
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360879 [3] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so.
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360892 [3] NCCL INFO [Proxy Service] Device 3 CPU core 20
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360893 [3] NCCL INFO [Proxy Service UDS] Device 3 CPU core 27
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so.
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 1
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360894 [0] NCCL INFO [Proxy Service] Device 0 CPU core 29
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360895 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 37
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO threadThresholds 8/8/64 | 32/8/64 | 512 | 512
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO 8 coll channels, 8 collnet channels, 0 nvls channels, 8 p2p channels, 2 p2p channels per peer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO threadThresholds 8/8/64 | 32/8/64 | 512 | 512
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO 8 coll channels, 8 collnet channels, 0 nvls channels, 8 p2p channels, 2 p2p channels per peer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360879 [3] NCCL INFO threadThresholds 8/8/64 | 32/8/64 | 512 | 512
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360879 [3] NCCL INFO 8 coll channels, 8 collnet channels, 0 nvls channels, 8 p2p channels, 2 p2p channels per peer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO threadThresholds 8/8/64 | 32/8/64 | 512 | 512
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO 8 coll channels, 8 collnet channels, 0 nvls channels, 8 p2p channels, 2 p2p channels per peer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO CC Off, workFifoBytes 1048576
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so. Using internal tuner plugin.
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v4 symbol.
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v3 symbol.
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v2 symbol, using internal tuner instead.
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO ncclCommInitAll comm 0x12242130 rank 1 nranks 4 cudaDev 1 nvmlDev 5 busId 600000 commId 0xa5c975f20549b316 - Init COMPLETE
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360877 [1] NCCL INFO Init timings - ncclCommInitAll: rank 1 nranks 4 total 0.53 (kernels 0.46, alloc 0.02, bootstrap 0.00, allgathers 0.00, topo 0.02, graphs 0.01, connections 0.02, rest 0.00)
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360879 [3] NCCL INFO ncclCommInitAll comm 0x12465150 rank 3 nranks 4 cudaDev 3 nvmlDev 7 busId 800000 commId 0xa5c975f20549b316 - Init COMPLETE
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360879 [3] NCCL INFO Init timings - ncclCommInitAll: rank 3 nranks 4 total 0.53 (kernels 0.46, alloc 0.02, bootstrap 0.00, allgathers 0.00, topo 0.02, graphs 0.01, connections 0.02, rest 0.01)
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO ncclCommInitAll comm 0x12353800 rank 2 nranks 4 cudaDev 2 nvmlDev 6 busId 700000 commId 0xa5c975f20549b316 - Init COMPLETE
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360878 [2] NCCL INFO Init timings - ncclCommInitAll: rank 2 nranks 4 total 0.53 (kernels 0.46, alloc 0.02, bootstrap 0.00, allgathers 0.00, topo 0.02, graphs 0.01, connections 0.02, rest 0.00)
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO ncclCommInitAll comm 0x12132bb0 rank 0 nranks 4 cudaDev 0 nvmlDev 4 busId 500000 commId 0xa5c975f20549b316 - Init COMPLETE
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360876 [0] NCCL INFO Init timings - ncclCommInitAll: rank 0 nranks 4 total 0.53 (kernels 0.46, alloc 0.02, bootstrap 0.00, allgathers 0.00, topo 0.02, graphs 0.01, connections 0.02, rest 0.00)
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360896 [3] NCCL INFO Channel 00/0 : 3[7] -> 0[4] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360898 [1] NCCL INFO Channel 00/0 : 1[5] -> 2[6] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360896 [3] NCCL INFO Channel 03/0 : 3[7] -> 0[4] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360898 [1] NCCL INFO Channel 02/0 : 1[5] -> 2[6] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360896 [3] NCCL INFO Channel 04/0 : 3[7] -> 0[4] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360897 [2] NCCL INFO Channel 00/0 : 2[6] -> 3[7] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360899 [0] NCCL INFO Channel 00/0 : 0[4] -> 1[5] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360898 [1] NCCL INFO Channel 04/0 : 1[5] -> 2[6] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360896 [3] NCCL INFO Channel 07/0 : 3[7] -> 0[4] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360897 [2] NCCL INFO Channel 04/0 : 2[6] -> 3[7] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360899 [0] NCCL INFO Channel 04/0 : 0[4] -> 1[5] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360898 [1] NCCL INFO Channel 06/0 : 1[5] -> 2[6] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360896 [3] NCCL INFO Channel 02/0 : 3[7] -> 1[5] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360899 [0] NCCL INFO Channel 03/0 : 0[4] -> 2[6] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360897 [2] NCCL INFO Channel 02/0 : 2[6] -> 0[4] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360898 [1] NCCL INFO Channel 03/0 : 1[5] -> 3[7] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360896 [3] NCCL INFO Channel 06/0 : 3[7] -> 1[5] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360899 [0] NCCL INFO Channel 07/0 : 0[4] -> 2[6] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360897 [2] NCCL INFO Channel 06/0 : 2[6] -> 0[4] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360898 [1] NCCL INFO Channel 07/0 : 1[5] -> 3[7] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360899 [0] NCCL INFO Channel 01/0 : 0[4] -> 3[7] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360897 [2] NCCL INFO Channel 01/0 : 2[6] -> 1[5] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360899 [0] NCCL INFO Channel 02/0 : 0[4] -> 3[7] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360897 [2] NCCL INFO Channel 03/0 : 2[6] -> 1[5] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360899 [0] NCCL INFO Channel 05/0 : 0[4] -> 3[7] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360897 [2] NCCL INFO Channel 05/0 : 2[6] -> 1[5] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360896 [3] NCCL INFO Channel 01/0 : 3[7] -> 2[6] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360898 [1] NCCL INFO Channel 01/0 : 1[5] -> 0[4] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360899 [0] NCCL INFO Channel 06/0 : 0[4] -> 3[7] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360897 [2] NCCL INFO Channel 07/0 : 2[6] -> 1[5] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360896 [3] NCCL INFO Channel 05/0 : 3[7] -> 2[6] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360898 [1] NCCL INFO Channel 05/0 : 1[5] -> 0[4] via P2P/direct pointer
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360899 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360897 [2] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360898 [1] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
7a05a34eb459462580c1cf15dccd48e000002L:2360481:2360896 [3] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
0%| | 1/1565 [00:03<1:25:15, 3.27s/it]
0%| | 2/1565 [00:04<49:45, 1.91s/it]
0%| | 3/1565 [00:05<38:26, 1.48s/it]
0%| | 4/1565 [00:06<33:10, 1.28s/it]
0%| | 5/1565 [00:07<30:12, 1.16s/it]
0%| | 6/1565 [00:08<28:24, 1.09s/it]
0%| | 7/1565 [00:09<27:17, 1.05s/it]
1%| | 8/1565 [00:10<26:33, 1.02s/it]
1%| | 9/1565 [00:10<26:02, 1.00s/it]
1%| | 10/1565 [00:11<25:41, 1.01it/s]
1%| | 11/1565 [00:12<25:27, 1.02it/s]
1%| | 12/1565 [00:13<25:16, 1.02it/s]
1%| | 13/1565 [00:14<25:09, 1.03it/s]
1%| | 14/1565 [00:15<25:02, 1.03it/s]
1%| | 15/1565 [00:16<25:00, 1.03it/s]
1%| | 16/1565 [00:17<24:56, 1.03it/s]
1%| | 17/1565 [00:18<24:54, 1.04it/s]
1%| | 18/1565 [00:19<24:52, 1.04it/s]
1%| | 19/1565 [00:20<24:52, 1.04it/s]
1%|โ | 20/1565 [00:21<24:52, 1.04it/s]
1%|โ | 21/1565 [00:22<24:48, 1.04it/s]
1%|โ | 22/1565 [00:23<24:48, 1.04it/s]
1%|โ | 23/1565 [00:24<24:48, 1.04it/s]
2%|โ | 24/1565 [00:25<24:47, 1.04it/s]
2%|โ | 25/1565 [00:26<24:45, 1.04it/s]
2%|โ | 26/1565 [00:27<24:46, 1.04it/s]
2%|โ | 27/1565 [00:28<24:45, 1.04it/s]
2%|โ | 28/1565 [00:29<24:46, 1.03it/s]
2%|โ | 29/1565 [00:30<24:44, 1.03it/s]
2%|โ | 30/1565 [00:31<24:45, 1.03it/s]
2%|โ | 31/1565 [00:32<24:46, 1.03it/s]
2%|โ | 32/1565 [00:33<24:45, 1.03it/s]
2%|โ | 33/1565 [00:34<24:43, 1.03it/s]
2%|โ | 34/1565 [00:35<24:44, 1.03it/s]
2%|โ | 35/1565 [00:36<24:46, 1.03it/s]
2%|โ | 36/1565 [00:37<24:47, 1.03it/s]
2%|โ | 37/1565 [00:38<24:46, 1.03it/s]
2%|โ | 38/1565 [00:39<24:45, 1.03it/s]
2%|โ | 39/1565 [00:39<24:45, 1.03it/s]
3%|โ | 40/1565 [00:40<24:41, 1.03it/s]
3%|โ | 41/1565 [00:41<24:40, 1.03it/s]
3%|โ | 42/1565 [00:42<24:40, 1.03it/s]
3%|โ | 43/1565 [00:43<24:40, 1.03it/s]
3%|โ | 44/1565 [00:44<24:38, 1.03it/s]
3%|โ | 45/1565 [00:45<24:34, 1.03it/s]
3%|โ | 46/1565 [00:46<24:37, 1.03it/s]
3%|โ | 47/1565 [00:48<26:28, 1.05s/it]
3%|โ | 48/1565 [00:48<25:52, 1.02s/it]
3%|โ | 49/1565 [00:49<25:28, 1.01s/it]
3%|โ | 50/1565 [00:50<25:14, 1.00it/s]
3%|โ | 51/1565 [00:51<25:00, 1.01it/s]
3%|โ | 52/1565 [00:52<24:51, 1.01it/s]
3%|โ | 53/1565 [00:53<24:43, 1.02it/s]
3%|โ | 54/1565 [00:54<24:38, 1.02it/s]
4%|โ | 55/1565 [00:55<24:35, 1.02it/s]
4%|โ | 56/1565 [00:56<24:32, 1.02it/s]
4%|โ | 57/1565 [00:57<24:29, 1.03it/s]
4%|โ | 58/1565 [00:58<24:28, 1.03it/s]
4%|โ | 59/1565 [00:59<24:27, 1.03it/s]
4%|โ | 60/1565 [01:00<24:27, 1.03it/s]
4%|โ | 61/1565 [01:01<24:28, 1.02it/s]
4%|โ | 62/1565 [01:02<24:26, 1.02it/s]
4%|โ | 63/1565 [01:03<24:25, 1.02it/s]
4%|โ | 64/1565 [01:04<24:24, 1.03it/s]
4%|โ | 65/1565 [01:05<24:23, 1.02it/s]
4%|โ | 66/1565 [01:06<24:22, 1.02it/s]
4%|โ | 67/1565 [01:07<24:20, 1.03it/s]
4%|โ | 68/1565 [01:08<24:20, 1.02it/s]
4%|โ | 69/1565 [01:09<24:22, 1.02it/s]
4%|โ | 70/1565 [01:10<24:20, 1.02it/s]
5%|โ | 71/1565 [01:11<24:17, 1.02it/s]
5%|โ | 72/1565 [01:12<24:15, 1.03it/s]
5%|โ | 73/1565 [01:13<24:14, 1.03it/s]
5%|โ | 74/1565 [01:14<24:15, 1.02it/s]
5%|โ | 75/1565 [01:15<24:11, 1.03it/s]
5%|โ | 76/1565 [01:16<24:10, 1.03it/s]
5%|โ | 77/1565 [01:17<24:09, 1.03it/s]
5%|โ | 78/1565 [01:18<24:09, 1.03it/s]
5%|โ | 79/1565 [01:19<24:08, 1.03it/s]
5%|โ | 80/1565 [01:20<24:07, 1.03it/s]
5%|โ | 81/1565 [01:21<24:06, 1.03it/s]
5%|โ | 82/1565 [01:22<24:05, 1.03it/s]
5%|โ | 83/1565 [01:23<24:06, 1.02it/s]
5%|โ | 84/1565 [01:24<24:06, 1.02it/s]
5%|โ | 85/1565 [01:25<24:05, 1.02it/s]
5%|โ | 86/1565 [01:26<24:03, 1.02it/s]
6%|โ | 87/1565 [01:27<24:06, 1.02it/s]
6%|โ | 88/1565 [01:27<24:05, 1.02it/s]
6%|โ | 89/1565 [01:28<24:03, 1.02it/s]
6%|โ | 90/1565 [01:29<24:01, 1.02it/s]
6%|โ | 91/1565 [01:30<23:58, 1.02it/s]
6%|โ | 92/1565 [01:31<23:58, 1.02it/s]
6%|โ | 93/1565 [01:32<23:55, 1.03it/s]
6%|โ | 94/1565 [01:34<25:41, 1.05s/it]
6%|โ | 95/1565 [01:35<25:07, 1.03s/it]
6%|โ | 96/1565 [01:36<24:45, 1.01s/it]
6%|โ | 97/1565 [01:37<24:29, 1.00s/it]
6%|โ | 98/1565 [01:37<24:18, 1.01it/s]
6%|โ | 99/1565 [01:38<24:10, 1.01it/s]
6%|โ | 100/1565 [01:39<24:03, 1.02it/s]
6%|โ | 101/1565 [01:40<24:00, 1.02it/s]
7%|โ | 102/1565 [01:41<24:00, 1.02it/s]
7%|โ | 103/1565 [01:42<23:55, 1.02it/s]
7%|โ | 104/1565 [01:43<23:51, 1.02it/s]
7%|โ | 105/1565 [01:44<23:48, 1.02it/s]
7%|โ | 106/1565 [01:45<23:47, 1.02it/s]
7%|โ | 107/1565 [01:46<23:48, 1.02it/s]
7%|โ | 108/1565 [01:47<23:45, 1.02it/s]
7%|โ | 109/1565 [01:48<23:44, 1.02it/s]
7%|โ | 110/1565 [01:49<23:42, 1.02it/s]
7%|โ | 111/1565 [01:50<23:38, 1.02it/s]
7%|โ | 112/1565 [01:51<23:39, 1.02it/s]
7%|โ | 113/1565 [01:52<23:36, 1.02it/s]
7%|โ | 114/1565 [01:53<23:36, 1.02it/s]
7%|โ | 115/1565 [01:54<23:34, 1.03it/s]
7%|โ | 116/1565 [01:55<23:33, 1.03it/s]
7%|โ | 117/1565 [01:56<23:33, 1.02it/s]
8%|โ | 118/1565 [01:57<23:32, 1.02it/s]
8%|โ | 119/1565 [01:58<23:31, 1.02it/s]
8%|โ | 120/1565 [01:59<23:31, 1.02it/s]
8%|โ | 121/1565 [02:00<23:30, 1.02it/s]
8%|โ | 122/1565 [02:01<23:27, 1.02it/s]
8%|โ | 123/1565 [02:02<23:26, 1.02it/s]
8%|โ | 124/1565 [02:03<23:25, 1.03it/s]
8%|โ | 125/1565 [02:04<23:25, 1.02it/s]
8%|โ | 126/1565 [02:05<23:24, 1.02it/s]
8%|โ | 127/1565 [02:06<23:28, 1.02it/s]
8%|โ | 128/1565 [02:07<23:26, 1.02it/s]
8%|โ | 129/1565 [02:08<23:24, 1.02it/s]
8%|โ | 130/1565 [02:09<23:23, 1.02it/s]
8%|โ | 131/1565 [02:10<23:19, 1.02it/s]
8%|โ | 132/1565 [02:11<23:17, 1.03it/s]
8%|โ | 133/1565 [02:12<23:18, 1.02it/s]
9%|โ | 134/1565 [02:13<23:16, 1.02it/s]
9%|โ | 135/1565 [02:14<23:14, 1.03it/s]
9%|โ | 136/1565 [02:15<23:12, 1.03it/s]
9%|โ | 137/1565 [02:16<23:13, 1.02it/s]
9%|โ | 138/1565 [02:17<23:13, 1.02it/s]
9%|โ | 139/1565 [02:18<23:10, 1.03it/s]
9%|โ | 140/1565 [02:19<23:07, 1.03it/s]
9%|โ | 141/1565 [02:20<24:43, 1.04s/it]
9%|โ | 142/1565 [02:21<24:13, 1.02s/it]
9%|โ | 143/1565 [02:22<23:52, 1.01s/it]
9%|โ | 144/1565 [02:23<23:38, 1.00it/s]
9%|โ | 145/1565 [02:24<23:29, 1.01it/s]
9%|โ | 146/1565 [02:25<23:21, 1.01it/s]
9%|โ | 147/1565 [02:26<23:14, 1.02it/s]
9%|โ | 148/1565 [02:27<23:10, 1.02it/s]
10%|โ | 149/1565 [02:28<23:08, 1.02it/s]
10%|โ | 150/1565 [02:28<23:09, 1.02it/s]
10%|โ | 151/1565 [02:29<23:10, 1.02it/s]
10%|โ | 152/1565 [02:30<23:07, 1.02it/s]
10%|โ | 153/1565 [02:31<23:04, 1.02it/s]
10%|โ | 154/1565 [02:32<23:02, 1.02it/s]
10%|โ | 155/1565 [02:33<23:02, 1.02it/s]
10%|โ | 156/1565 [02:34<22:59, 1.02it/s]
10%|โ | 157/1565 [02:35<22:58, 1.02it/s]
10%|โ | 158/1565 [02:36<22:54, 1.02it/s]
10%|โ | 159/1565 [02:37<22:53, 1.02it/s]
10%|โ | 160/1565 [02:38<22:51, 1.02it/s]
10%|โ | 161/1565 [02:39<22:48, 1.03it/s]
10%|โ | 162/1565 [02:40<22:51, 1.02it/s]
10%|โ | 163/1565 [02:41<22:51, 1.02it/s]
10%|โ | 164/1565 [02:42<22:49, 1.02it/s]
11%|โ | 165/1565 [02:43<22:50, 1.02it/s]
11%|โ | 166/1565 [02:44<22:50, 1.02it/s]
11%|โ | 167/1565 [02:45<22:50, 1.02it/s]
11%|โ | 168/1565 [02:46<22:47, 1.02it/s]
11%|โ | 169/1565 [02:47<22:48, 1.02it/s]
11%|โ | 170/1565 [02:48<22:47, 1.02it/s]
11%|โ | 171/1565 [02:49<22:46, 1.02it/s]
11%|โ | 172/1565 [02:50<22:44, 1.02it/s]
11%|โ | 173/1565 [02:51<22:43, 1.02it/s]
11%|โ | 174/1565 [02:52<22:41, 1.02it/s]
11%|โ | 175/1565 [02:53<22:39, 1.02it/s]
11%|โ | 176/1565 [02:54<22:37, 1.02it/s]
11%|โโ | 177/1565 [02:55<22:38, 1.02it/s]
11%|โโ | 178/1565 [02:56<22:36, 1.02it/s]
11%|โโ | 179/1565 [02:57<22:35, 1.02it/s]
12%|โโ | 180/1565 [02:58<22:35, 1.02it/s]
12%|โโ | 181/1565 [02:59<22:33, 1.02it/s]
12%|โโ | 182/1565 [03:00<22:30, 1.02it/s]
12%|โโ | 183/1565 [03:01<22:32, 1.02it/s]
12%|โโ | 184/1565 [03:02<22:31, 1.02it/s]
12%|โโ | 185/1565 [03:03<22:27, 1.02it/s]
12%|โโ | 186/1565 [03:04<22:25, 1.02it/s]
12%|โโ | 187/1565 [03:05<22:24, 1.03it/s]
12%|โโ | 188/1565 [03:06<23:55, 1.04s/it]
12%|โโ | 189/1565 [03:07<23:25, 1.02s/it]
12%|โโ | 190/1565 [03:08<23:08, 1.01s/it]
12%|โโ | 191/1565 [03:09<22:54, 1.00s/it]
12%|โโ | 192/1565 [03:10<22:44, 1.01it/s]
12%|โโ | 193/1565 [03:11<22:33, 1.01it/s]
12%|โโ | 194/1565 [03:12<22:28, 1.02it/s]
12%|โโ | 195/1565 [03:13<22:25, 1.02it/s]
13%|โโ | 196/1565 [03:14<22:22, 1.02it/s]
13%|โโ | 197/1565 [03:15<22:20, 1.02it/s]
13%|โโ | 198/1565 [03:16<22:16, 1.02it/s]
13%|โโ | 199/1565 [03:17<22:13, 1.02it/s]
13%|โโ | 200/1565 [03:18<22:11, 1.03it/s]
13%|โโ | 201/1565 [03:19<22:11, 1.02it/s]
13%|โโ | 202/1565 [03:20<22:11, 1.02it/s]
13%|โโ | 203/1565 [03:21<22:11, 1.02it/s]
13%|โโ | 204/1565 [03:22<22:08, 1.02it/s]
13%|โโ | 205/1565 [03:22<22:05, 1.03it/s]
13%|โโ | 206/1565 [03:23<22:05, 1.03it/s]
13%|โโ | 207/1565 [03:24<22:07, 1.02it/s]
13%|โโ | 208/1565 [03:25<22:06, 1.02it/s]
13%|โโ | 209/1565 [03:26<22:05, 1.02it/s]
13%|โโ | 210/1565 [03:27<22:03, 1.02it/s]
13%|โโ | 211/1565 [03:28<22:01, 1.02it/s]
14%|โโ | 212/1565 [03:29<21:59, 1.03it/s]
14%|โโ | 213/1565 [03:30<22:01, 1.02it/s]
14%|โโ | 214/1565 [03:31<22:00, 1.02it/s]
14%|โโ | 215/1565 [03:32<21:57, 1.02it/s]
14%|โโ | 216/1565 [03:33<21:58, 1.02it/s]
14%|โโ | 217/1565 [03:34<21:56, 1.02it/s]
14%|โโ | 218/1565 [03:35<21:58, 1.02it/s]
14%|โโ | 219/1565 [03:36<21:55, 1.02it/s]
14%|โโ | 220/1565 [03:37<21:53, 1.02it/s]
14%|โโ | 221/1565 [03:38<21:52, 1.02it/s]
14%|โโ | 222/1565 [03:39<21:50, 1.02it/s]
14%|โโ | 223/1565 [03:40<21:51, 1.02it/s]
14%|โโ | 224/1565 [03:41<21:50, 1.02it/s]
14%|โโ | 225/1565 [03:42<21:50, 1.02it/s]
14%|โโ | 226/1565 [03:43<21:47, 1.02it/s]
15%|โโ | 227/1565 [03:44<21:46, 1.02it/s]
15%|โโ | 228/1565 [03:45<21:46, 1.02it/s]
15%|โโ | 229/1565 [03:46<21:45, 1.02it/s]
15%|โโ | 230/1565 [03:47<21:43, 1.02it/s]
15%|โโ | 231/1565 [03:48<21:40, 1.03it/s]
15%|โโ | 232/1565 [03:49<21:40, 1.02it/s]
15%|โโ | 233/1565 [03:50<21:41, 1.02it/s]
15%|โโ | 234/1565 [03:51<21:43, 1.02it/s]
15%|โโ | 235/1565 [03:52<23:13, 1.05s/it]
15%|โโ | 236/1565 [03:53<22:45, 1.03s/it]
15%|โโ | 237/1565 [03:54<22:24, 1.01s/it]
15%|โโ | 238/1565 [03:55<22:08, 1.00s/it]
15%|โโ | 239/1565 [03:56<21:56, 1.01it/s]
15%|โโ | 240/1565 [03:57<21:49, 1.01it/s]
15%|โโ | 241/1565 [03:58<21:45, 1.01it/s]
15%|โโ | 242/1565 [03:59<21:42, 1.02it/s]
16%|โโ | 243/1565 [04:00<21:39, 1.02it/s]
16%|โโ | 244/1565 [04:01<21:35, 1.02it/s]
16%|โโ | 245/1565 [04:02<21:32, 1.02it/s]
16%|โโ | 246/1565 [04:03<21:31, 1.02it/s]
16%|โโ | 247/1565 [04:04<21:30, 1.02it/s]
16%|โโ | 248/1565 [04:05<21:27, 1.02it/s]
16%|โโ | 249/1565 [04:06<21:27, 1.02it/s]
16%|โโ | 250/1565 [04:07<21:25, 1.02it/s]
16%|โโ | 251/1565 [04:08<21:24, 1.02it/s]
16%|โโ | 252/1565 [04:09<21:21, 1.02it/s]
16%|โโ | 253/1565 [04:10<21:23, 1.02it/s]
16%|โโ | 254/1565 [04:11<21:20, 1.02it/s]
16%|โโ | 255/1565 [04:12<21:20, 1.02it/s]
16%|โโ | 256/1565 [04:13<21:20, 1.02it/s]
16%|โโ | 257/1565 [04:14<21:21, 1.02it/s]
16%|โโ | 258/1565 [04:15<21:19, 1.02it/s]
17%|โโ | 259/1565 [04:15<21:18, 1.02it/s]
17%|โโ | 260/1565 [04:16<21:14, 1.02it/s]
17%|โโ | 261/1565 [04:17<21:13, 1.02it/s]
17%|โโ | 262/1565 [04:18<21:12, 1.02it/s]
17%|โโ | 263/1565 [04:19<21:14, 1.02it/s]
17%|โโ | 264/1565 [04:20<21:10, 1.02it/s]
17%|โโ | 265/1565 [04:21<21:07, 1.03it/s]
17%|โโ | 266/1565 [04:22<21:07, 1.03it/s]
17%|โโ | 267/1565 [04:23<21:07, 1.02it/s]
17%|โโ | 268/1565 [04:24<21:07, 1.02it/s]
17%|โโ | 269/1565 [04:25<21:05, 1.02it/s]
17%|โโ | 270/1565 [04:26<21:02, 1.03it/s]
17%|โโ | 271/1565 [04:27<21:01, 1.03it/s]
17%|โโ | 272/1565 [04:28<21:00, 1.03it/s]
17%|โโ | 273/1565 [04:29<20:59, 1.03it/s]
18%|โโ | 274/1565 [04:30<20:59, 1.03it/s]
18%|โโ | 275/1565 [04:31<20:57, 1.03it/s]
18%|โโ | 276/1565 [04:32<20:56, 1.03it/s]
18%|โโ | 277/1565 [04:33<20:55, 1.03it/s]
18%|โโ | 278/1565 [04:34<20:54, 1.03it/s]
18%|โโ | 279/1565 [04:35<20:55, 1.02it/s]
18%|โโ | 280/1565 [04:36<20:53, 1.02it/s]
18%|โโ | 281/1565 [04:37<20:52, 1.03it/s]
18%|โโ | 282/1565 [04:38<22:26, 1.05s/it]
18%|โโ | 283/1565 [04:39<21:57, 1.03s/it]
18%|โโ | 284/1565 [04:40<21:36, 1.01s/it]
18%|โโ | 285/1565 [04:41<21:21, 1.00s/it]
18%|โโ | 286/1565 [04:42<21:11, 1.01it/s]
18%|โโ | 287/1565 [04:43<21:07, 1.01it/s]
18%|โโ | 288/1565 [04:44<21:00, 1.01it/s]
18%|โโ | 289/1565 [04:45<20:54, 1.02it/s]
19%|โโ | 290/1565 [04:46<20:51, 1.02it/s]
19%|โโ | 291/1565 [04:47<20:50, 1.02it/s]
19%|โโ | 292/1565 [04:48<20:48, 1.02it/s]
19%|โโ | 293/1565 [04:49<20:48, 1.02it/s]
19%|โโ | 294/1565 [04:50<20:45, 1.02it/s]
19%|โโ | 295/1565 [04:51<20:42, 1.02it/s]
19%|โโ | 296/1565 [04:52<20:39, 1.02it/s]
19%|โโ | 297/1565 [04:53<20:38, 1.02it/s]
19%|โโ | 298/1565 [04:54<20:39, 1.02it/s]
19%|โโ | 299/1565 [04:55<20:37, 1.02it/s]
19%|โโ | 300/1565 [04:56<20:34, 1.02it/s]
19%|โโ | 301/1565 [04:57<20:31, 1.03it/s]
19%|โโ | 302/1565 [04:58<20:31, 1.03it/s]
19%|โโ | 303/1565 [04:59<20:34, 1.02it/s]
19%|โโ | 304/1565 [05:00<20:35, 1.02it/s]
19%|โโ | 305/1565 [05:01<20:34, 1.02it/s]
20%|โโ | 306/1565 [05:02<20:31, 1.02it/s]
20%|โโ | 307/1565 [05:03<20:30, 1.02it/s]
20%|โโ | 308/1565 [05:04<20:28, 1.02it/s]
20%|โโ | 309/1565 [05:05<20:27, 1.02it/s]
20%|โโ | 310/1565 [05:06<20:27, 1.02it/s]
20%|โโ | 311/1565 [05:07<20:27, 1.02it/s]
20%|โโ | 312/1565 [05:07<20:11, 1.03it/s]
20%|โโ | 313/1565 [05:08<17:11, 1.21it/s]
20%|โโ | 314/1565 [05:09<18:39, 1.12it/s]
20%|โโ | 315/1565 [05:10<19:08, 1.09it/s]
20%|โโ | 316/1565 [05:11<19:31, 1.07it/s]
20%|โโ | 317/1565 [05:12<19:46, 1.05it/s]
20%|โโ | 318/1565 [05:13<19:52, 1.05it/s]
20%|โโ | 319/1565 [05:14<19:57, 1.04it/s]
20%|โโ | 320/1565 [05:15<20:02, 1.03it/s]
21%|โโ | 321/1565 [05:16<20:04, 1.03it/s]
21%|โโ | 322/1565 [05:17<20:06, 1.03it/s]
21%|โโ | 323/1565 [05:18<20:07, 1.03it/s]
21%|โโ | 324/1565 [05:19<20:07, 1.03it/s]
21%|โโ | 325/1565 [05:20<20:09, 1.03it/s]
21%|โโ | 326/1565 [05:21<20:10, 1.02it/s]
21%|โโ | 327/1565 [05:22<20:08, 1.02it/s]
21%|โโ | 328/1565 [05:23<20:06, 1.03it/s]
21%|โโ | 329/1565 [05:24<21:30, 1.04s/it]
21%|โโ | 330/1565 [05:25<21:04, 1.02s/it]
21%|โโ | 331/1565 [05:26<20:44, 1.01s/it]
21%|โโ | 332/1565 [05:27<20:32, 1.00it/s]
21%|โโโ | 333/1565 [05:28<20:23, 1.01it/s]
21%|โโโ | 334/1565 [05:29<20:18, 1.01it/s]
21%|โโโ | 335/1565 [05:30<20:11, 1.01it/s]
21%|โโโ | 336/1565 [05:31<20:07, 1.02it/s]
22%|โโโ | 337/1565 [05:32<20:04, 1.02it/s]
22%|โโโ | 338/1565 [05:33<20:03, 1.02it/s]
22%|โโโ | 339/1565 [05:34<20:01, 1.02it/s]
22%|โโโ | 340/1565 [05:35<19:58, 1.02it/s]
22%|โโโ | 341/1565 [05:36<19:55, 1.02it/s]
22%|โโโ | 342/1565 [05:37<19:54, 1.02it/s]
22%|โโโ | 343/1565 [05:38<19:55, 1.02it/s]
22%|โโโ | 344/1565 [05:39<19:56, 1.02it/s]
22%|โโโ | 345/1565 [05:40<19:54, 1.02it/s]
22%|โโโ | 346/1565 [05:41<19:53, 1.02it/s]
22%|โโโ | 347/1565 [05:41<19:50, 1.02it/s]
22%|โโโ | 348/1565 [05:42<19:48, 1.02it/s]
22%|โโโ | 349/1565 [05:43<19:47, 1.02it/s]
22%|โโโ | 350/1565 [05:44<19:46, 1.02it/s]
22%|โโโ | 351/1565 [05:45<19:49, 1.02it/s]
22%|โโโ | 352/1565 [05:46<19:47, 1.02it/s]
23%|โโโ | 353/1565 [05:47<19:43, 1.02it/s]
23%|โโโ | 354/1565 [05:48<19:41, 1.02it/s]
23%|โโโ | 355/1565 [05:49<19:41, 1.02it/s]
23%|โโโ | 356/1565 [05:50<19:41, 1.02it/s]
23%|โโโ | 357/1565 [05:51<19:40, 1.02it/s]
23%|โโโ | 358/1565 [05:52<19:39, 1.02it/s]
23%|โโโ | 359/1565 [05:53<19:36, 1.03it/s]
23%|โโโ | 360/1565 [05:54<19:36, 1.02it/s]
23%|โโโ | 361/1565 [05:55<19:36, 1.02it/s]
23%|โโโ | 362/1565 [05:56<19:34, 1.02it/s]
23%|โโโ | 363/1565 [05:57<19:32, 1.02it/s]
23%|โโโ | 364/1565 [05:58<19:32, 1.02it/s]
23%|โโโ | 365/1565 [05:59<19:30, 1.02it/s]
23%|โโโ | 366/1565 [06:00<19:29, 1.03it/s]
23%|โโโ | 367/1565 [06:01<19:31, 1.02it/s]
24%|โโโ | 368/1565 [06:02<19:28, 1.02it/s]
24%|โโโ | 369/1565 [06:03<19:27, 1.02it/s]
24%|โโโ | 370/1565 [06:04<19:26, 1.02it/s]
24%|โโโ | 371/1565 [06:05<19:25, 1.02it/s]
24%|โโโ | 372/1565 [06:06<19:26, 1.02it/s]
24%|โโโ | 373/1565 [06:07<19:23, 1.02it/s]
24%|โโโ | 374/1565 [06:08<19:22, 1.02it/s]
24%|โโโ | 375/1565 [06:09<19:19, 1.03it/s]
24%|โโโ | 376/1565 [06:10<20:39, 1.04s/it]
24%|โโโ | 377/1565 [06:11<20:14, 1.02s/it]
24%|โโโ | 378/1565 [06:12<19:56, 1.01s/it]
24%|โโโ | 379/1565 [06:13<19:46, 1.00s/it]
24%|โโโ | 380/1565 [06:14<19:38, 1.01it/s]
24%|โโโ | 381/1565 [06:15<19:31, 1.01it/s]
24%|โโโ | 382/1565 [06:16<19:24, 1.02it/s]
24%|โโโ | 383/1565 [06:17<19:19, 1.02it/s]
25%|โโโ | 384/1565 [06:18<19:17, 1.02it/s]
25%|โโโ | 385/1565 [06:19<19:17, 1.02it/s]
25%|โโโ | 386/1565 [06:20<19:16, 1.02it/s]
25%|โโโ | 387/1565 [06:21<19:11, 1.02it/s]
25%|โโโ | 388/1565 [06:22<19:09, 1.02it/s]
25%|โโโ | 389/1565 [06:23<19:07, 1.03it/s]
25%|โโโ | 390/1565 [06:24<19:06, 1.02it/s]
25%|โโโ | 391/1565 [06:25<19:06, 1.02it/s]
25%|โโโ | 392/1565 [06:26<19:06, 1.02it/s]
25%|โโโ | 393/1565 [06:27<19:05, 1.02it/s]
25%|โโโ | 394/1565 [06:28<19:06, 1.02it/s]
25%|โโโ | 395/1565 [06:29<19:03, 1.02it/s]
25%|โโโ | 396/1565 [06:30<19:00, 1.02it/s]
25%|โโโ | 397/1565 [06:31<19:00, 1.02it/s]
25%|โโโ | 398/1565 [06:32<18:58, 1.03it/s]
25%|โโโ | 399/1565 [06:32<18:57, 1.03it/s]
26%|โโโ | 400/1565 [06:33<18:56, 1.03it/s]
26%|โโโ | 401/1565 [06:34<18:54, 1.03it/s]
26%|โโโ | 402/1565 [06:35<18:54, 1.03it/s]
26%|โโโ | 403/1565 [06:36<18:52, 1.03it/s]
26%|โโโ | 404/1565 [06:37<18:55, 1.02it/s]
26%|โโโ | 405/1565 [06:38<18:56, 1.02it/s]
26%|โโโ | 406/1565 [06:39<18:54, 1.02it/s]
26%|โโโ | 407/1565 [06:40<18:53, 1.02it/s]
26%|โโโ | 408/1565 [06:41<18:54, 1.02it/s]
26%|โโโ | 409/1565 [06:42<18:51, 1.02it/s]
26%|โโโ | 410/1565 [06:43<18:52, 1.02it/s]
26%|โโโ | 411/1565 [06:44<18:51, 1.02it/s]
26%|โโโ | 412/1565 [06:45<18:49, 1.02it/s]
26%|โโโ | 413/1565 [06:46<18:50, 1.02it/s]
26%|โโโ | 414/1565 [06:47<18:48, 1.02it/s]
27%|โโโ | 415/1565 [06:48<18:45, 1.02it/s]
27%|โโโ | 416/1565 [06:49<18:44, 1.02it/s]
27%|โโโ | 417/1565 [06:50<18:43, 1.02it/s]
27%|โโโ | 418/1565 [06:51<18:43, 1.02it/s]
27%|โโโ | 419/1565 [06:52<18:41, 1.02it/s]
27%|โโโ | 420/1565 [06:53<18:38, 1.02it/s]
27%|โโโ | 421/1565 [06:54<18:36, 1.02it/s]
27%|โโโ | 422/1565 [06:55<18:36, 1.02it/s]
27%|โโโ | 423/1565 [06:56<19:59, 1.05s/it]
27%|โโโ | 424/1565 [06:57<19:33, 1.03s/it]
27%|โโโ | 425/1565 [06:58<19:15, 1.01s/it]
27%|โโโ | 426/1565 [06:59<19:02, 1.00s/it]
27%|โโโ | 427/1565 [07:00<18:52, 1.01it/s]
27%|โโโ | 428/1565 [07:01<18:45, 1.01it/s]
27%|โโโ | 429/1565 [07:02<18:39, 1.01it/s]
27%|โโโ | 430/1565 [07:03<18:36, 1.02it/s]
28%|โโโ | 431/1565 [07:04<18:34, 1.02it/s]
28%|โโโ | 432/1565 [07:05<18:32, 1.02it/s]
28%|โโโ | 433/1565 [07:06<18:29, 1.02it/s]
28%|โโโ | 434/1565 [07:07<18:27, 1.02it/s]
28%|โโโ | 435/1565 [07:08<18:26, 1.02it/s]
28%|โโโ | 436/1565 [07:09<18:24, 1.02it/s]
28%|โโโ | 437/1565 [07:10<18:24, 1.02it/s]
28%|โโโ | 438/1565 [07:11<18:22, 1.02it/s]
28%|โโโ | 439/1565 [07:12<18:19, 1.02it/s]
28%|โโโ | 440/1565 [07:13<18:17, 1.03it/s]
28%|โโโ | 441/1565 [07:14<18:17, 1.02it/s]
28%|โโโ | 442/1565 [07:15<18:16, 1.02it/s]
28%|โโโ | 443/1565 [07:16<18:15, 1.02it/s]
28%|โโโ | 444/1565 [07:17<18:14, 1.02it/s]
28%|โโโ | 445/1565 [07:18<18:14, 1.02it/s]
28%|โโโ | 446/1565 [07:19<18:15, 1.02it/s]
29%|โโโ | 447/1565 [07:20<18:15, 1.02it/s]
29%|โโโ | 448/1565 [07:21<18:14, 1.02it/s]
29%|โโโ | 449/1565 [07:22<18:10, 1.02it/s]
29%|โโโ | 450/1565 [07:23<18:08, 1.02it/s]
29%|โโโ | 451/1565 [07:24<18:05, 1.03it/s]
29%|โโโ | 452/1565 [07:25<18:05, 1.03it/s]
29%|โโโ | 453/1565 [07:26<18:02, 1.03it/s]
29%|โโโ | 454/1565 [07:26<18:01, 1.03it/s]
29%|โโโ | 455/1565 [07:27<18:01, 1.03it/s]
29%|โโโ | 456/1565 [07:28<17:59, 1.03it/s]
29%|โโโ | 457/1565 [07:29<17:59, 1.03it/s]
29%|โโโ | 458/1565 [07:30<17:59, 1.03it/s]
29%|โโโ | 459/1565 [07:31<17:57, 1.03it/s]
29%|โโโ | 460/1565 [07:32<17:56, 1.03it/s]
29%|โโโ | 461/1565 [07:33<17:54, 1.03it/s]
30%|โโโ | 462/1565 [07:34<17:51, 1.03it/s]
30%|โโโ | 463/1565 [07:35<17:51, 1.03it/s]
30%|โโโ | 464/1565 [07:36<17:50, 1.03it/s]
30%|โโโ | 465/1565 [07:37<17:50, 1.03it/s]
30%|โโโ | 466/1565 [07:38<17:50, 1.03it/s]
30%|โโโ | 467/1565 [07:39<17:51, 1.02it/s]
30%|โโโ | 468/1565 [07:40<17:49, 1.03it/s]
30%|โโโ | 469/1565 [07:41<17:46, 1.03it/s]
30%|โโโ | 470/1565 [07:42<19:00, 1.04s/it]
30%|โโโ | 471/1565 [07:43<18:36, 1.02s/it]
30%|โโโ | 472/1565 [07:44<18:19, 1.01s/it]
30%|โโโ | 473/1565 [07:45<18:06, 1.01it/s]
30%|โโโ | 474/1565 [07:46<17:55, 1.01it/s]
30%|โโโ | 475/1565 [07:47<17:49, 1.02it/s]
30%|โโโ | 476/1565 [07:48<17:46, 1.02it/s]
30%|โโโ | 477/1565 [07:49<17:43, 1.02it/s]
31%|โโโ | 478/1565 [07:50<17:40, 1.03it/s]
31%|โโโ | 479/1565 [07:51<17:37, 1.03it/s]
31%|โโโ | 480/1565 [07:52<17:35, 1.03it/s]
31%|โโโ | 481/1565 [07:53<17:33, 1.03it/s]
31%|โโโ | 482/1565 [07:54<17:32, 1.03it/s]
31%|โโโ | 483/1565 [07:55<17:32, 1.03it/s]
31%|โโโ | 484/1565 [07:56<17:30, 1.03it/s]
31%|โโโ | 485/1565 [07:57<17:29, 1.03it/s]
31%|โโโ | 486/1565 [07:58<17:29, 1.03it/s]
31%|โโโ | 487/1565 [07:59<17:29, 1.03it/s]
31%|โโโ | 488/1565 [08:00<17:29, 1.03it/s]
31%|โโโ | 489/1565 [08:01<17:27, 1.03it/s]
31%|โโโโ | 490/1565 [08:02<17:26, 1.03it/s]
31%|โโโโ | 491/1565 [08:03<17:26, 1.03it/s]
31%|โโโโ | 492/1565 [08:04<17:24, 1.03it/s]
32%|โโโโ | 493/1565 [08:05<17:22, 1.03it/s]
32%|โโโโ | 494/1565 [08:06<17:21, 1.03it/s]
32%|โโโโ | 495/1565 [08:07<17:22, 1.03it/s]
32%|โโโโ | 496/1565 [08:08<17:22, 1.03it/s]
32%|โโโโ | 497/1565 [08:09<17:21, 1.03it/s]
32%|โโโโ | 498/1565 [08:10<17:19, 1.03it/s]
32%|โโโโ | 499/1565 [08:11<17:17, 1.03it/s]
32%|โโโโ | 500/1565 [08:11<17:16, 1.03it/s]
32%|โโโโ | 501/1565 [08:12<17:15, 1.03it/s]
32%|โโโโ | 502/1565 [08:13<17:16, 1.03it/s]
32%|โโโโ | 503/1565 [08:14<17:13, 1.03it/s]
32%|โโโโ | 504/1565 [08:15<17:13, 1.03it/s]
32%|โโโโ | 505/1565 [08:16<17:13, 1.03it/s]
32%|โโโโ | 506/1565 [08:17<17:12, 1.03it/s]
32%|โโโโ | 507/1565 [08:18<17:11, 1.03it/s]
32%|โโโโ | 508/1565 [08:19<17:09, 1.03it/s]
33%|โโโโ | 509/1565 [08:20<17:07, 1.03it/s]
33%|โโโโ | 510/1565 [08:21<17:06, 1.03it/s]
33%|โโโโ | 511/1565 [08:22<17:05, 1.03it/s]
33%|โโโโ | 512/1565 [08:23<17:06, 1.03it/s]
33%|โโโโ | 513/1565 [08:24<17:03, 1.03it/s]
33%|โโโโ | 514/1565 [08:25<17:01, 1.03it/s]
33%|โโโโ | 515/1565 [08:26<16:59, 1.03it/s]
33%|โโโโ | 516/1565 [08:27<16:58, 1.03it/s]
33%|โโโโ | 517/1565 [08:28<18:11, 1.04s/it]
33%|โโโโ | 518/1565 [08:29<17:49, 1.02s/it]
33%|โโโโ | 519/1565 [08:30<17:31, 1.01s/it]
33%|โโโโ | 520/1565 [08:31<17:20, 1.00it/s]
33%|โโโโ | 521/1565 [08:32<17:10, 1.01it/s]
33%|โโโโ | 522/1565 [08:33<17:04, 1.02it/s]
33%|โโโโ | 523/1565 [08:34<17:00, 1.02it/s]
33%|โโโโ | 524/1565 [08:35<16:56, 1.02it/s]
34%|โโโโ | 525/1565 [08:36<16:55, 1.02it/s]
34%|โโโโ | 526/1565 [08:37<16:54, 1.02it/s]
34%|โโโโ | 527/1565 [08:38<16:54, 1.02it/s]
34%|โโโโ | 528/1565 [08:39<16:51, 1.03it/s]
34%|โโโโ | 529/1565 [08:40<16:49, 1.03it/s]
34%|โโโโ | 530/1565 [08:41<16:48, 1.03it/s]
34%|โโโโ | 531/1565 [08:42<16:46, 1.03it/s]
34%|โโโโ | 532/1565 [08:43<16:45, 1.03it/s]
34%|โโโโ | 533/1565 [08:44<16:43, 1.03it/s]
34%|โโโโ | 534/1565 [08:45<16:41, 1.03it/s]
34%|โโโโ | 535/1565 [08:46<16:40, 1.03it/s]
34%|โโโโ | 536/1565 [08:47<16:39, 1.03it/s]
34%|โโโโ | 537/1565 [08:48<16:37, 1.03it/s]
34%|โโโโ | 538/1565 [08:49<16:35, 1.03it/s]
34%|โโโโ | 539/1565 [08:50<16:34, 1.03it/s]
35%|โโโโ | 540/1565 [08:51<16:35, 1.03it/s]
35%|โโโโ | 541/1565 [08:52<16:33, 1.03it/s]
35%|โโโโ | 542/1565 [08:53<16:31, 1.03it/s]
35%|โโโโ | 543/1565 [08:54<16:29, 1.03it/s]
35%|โโโโ | 544/1565 [08:54<16:29, 1.03it/s]
35%|โโโโ | 545/1565 [08:55<16:30, 1.03it/s]
35%|โโโโ | 546/1565 [08:56<16:31, 1.03it/s]
35%|โโโโ | 547/1565 [08:57<16:28, 1.03it/s]
35%|โโโโ | 548/1565 [08:58<16:27, 1.03it/s]
35%|โโโโ | 549/1565 [08:59<16:27, 1.03it/s]
35%|โโโโ | 550/1565 [09:00<16:28, 1.03it/s]
35%|โโโโ | 551/1565 [09:01<16:29, 1.02it/s]
35%|โโโโ | 552/1565 [09:02<16:29, 1.02it/s]
35%|โโโโ | 553/1565 [09:03<16:43, 1.01it/s]
35%|โโโโ | 554/1565 [09:04<16:38, 1.01it/s]
35%|โโโโ | 555/1565 [09:05<16:32, 1.02it/s]
36%|โโโโ | 556/1565 [09:06<16:27, 1.02it/s]
36%|โโโโ | 557/1565 [09:07<16:24, 1.02it/s]
36%|โโโโ | 558/1565 [09:08<16:22, 1.02it/s]
36%|โโโโ | 559/1565 [09:09<16:20, 1.03it/s]
36%|โโโโ | 560/1565 [09:10<16:20, 1.03it/s]
36%|โโโโ | 561/1565 [09:11<16:21, 1.02it/s]
36%|โโโโ | 562/1565 [09:12<16:22, 1.02it/s]
36%|โโโโ | 563/1565 [09:13<16:20, 1.02it/s]
36%|โโโโ | 564/1565 [09:14<17:49, 1.07s/it]
36%|โโโโ | 565/1565 [09:15<17:21, 1.04s/it]
36%|โโโโ | 566/1565 [09:16<17:00, 1.02s/it]
36%|โโโโ | 567/1565 [09:17<16:44, 1.01s/it]
36%|โโโโ | 568/1565 [09:18<16:34, 1.00it/s]
36%|โโโโ | 569/1565 [09:19<16:30, 1.01it/s]
36%|โโโโ | 570/1565 [09:20<16:24, 1.01it/s]
36%|โโโโ | 571/1565 [09:21<16:20, 1.01it/s]
37%|โโโโ | 572/1565 [09:22<16:16, 1.02it/s]
37%|โโโโ | 573/1565 [09:23<16:10, 1.02it/s]
37%|โโโโ | 574/1565 [09:24<16:06, 1.03it/s]
37%|โโโโ | 575/1565 [09:25<16:04, 1.03it/s]
37%|โโโโ | 576/1565 [09:26<16:04, 1.02it/s]
37%|โโโโ | 577/1565 [09:27<16:19, 1.01it/s]
37%|โโโโ | 578/1565 [09:28<16:13, 1.01it/s]
37%|โโโโ | 579/1565 [09:29<16:11, 1.02it/s]
37%|โโโโ | 580/1565 [09:30<16:06, 1.02it/s]
37%|โโโโ | 581/1565 [09:31<16:02, 1.02it/s]
37%|โโโโ | 582/1565 [09:32<15:59, 1.02it/s]
37%|โโโโ | 583/1565 [09:33<15:59, 1.02it/s]
37%|โโโโ | 584/1565 [09:34<15:56, 1.03it/s]
37%|โโโโ | 585/1565 [09:35<15:54, 1.03it/s]
37%|โโโโ | 586/1565 [09:36<15:51, 1.03it/s]
38%|โโโโ | 587/1565 [09:37<15:50, 1.03it/s]
38%|โโโโ | 588/1565 [09:38<15:50, 1.03it/s]
38%|โโโโ | 589/1565 [09:39<15:51, 1.03it/s]
38%|โโโโ | 590/1565 [09:40<15:52, 1.02it/s]
38%|โโโโ | 591/1565 [09:41<15:49, 1.03it/s]
38%|โโโโ | 592/1565 [09:42<15:47, 1.03it/s]
38%|โโโโ | 593/1565 [09:43<15:45, 1.03it/s]
38%|โโโโ | 594/1565 [09:44<15:43, 1.03it/s]
38%|โโโโ | 595/1565 [09:45<15:42, 1.03it/s]
38%|โโโโ | 596/1565 [09:46<15:42, 1.03it/s]
38%|โโโโ | 597/1565 [09:47<15:43, 1.03it/s]
38%|โโโโ | 598/1565 [09:48<15:41, 1.03it/s]
38%|โโโโ | 599/1565 [09:48<15:39, 1.03it/s]
38%|โโโโ | 600/1565 [09:49<15:37, 1.03it/s]
38%|โโโโ | 601/1565 [09:50<15:37, 1.03it/s]
38%|โโโโ | 602/1565 [09:51<15:36, 1.03it/s]
39%|โโโโ | 603/1565 [09:52<15:35, 1.03it/s]
39%|โโโโ | 604/1565 [09:53<15:33, 1.03it/s]
39%|โโโโ | 605/1565 [09:54<15:31, 1.03it/s]
39%|โโโโ | 606/1565 [09:55<15:31, 1.03it/s]
39%|โโโโ | 607/1565 [09:56<15:30, 1.03it/s]
39%|โโโโ | 608/1565 [09:57<15:28, 1.03it/s]
39%|โโโโ | 609/1565 [09:58<15:27, 1.03it/s]
39%|โโโโ | 610/1565 [09:59<15:25, 1.03it/s]
39%|โโโโ | 611/1565 [10:00<16:38, 1.05s/it]
39%|โโโโ | 612/1565 [10:01<16:14, 1.02s/it]
39%|โโโโ | 613/1565 [10:02<15:59, 1.01s/it]
39%|โโโโ | 614/1565 [10:03<15:47, 1.00it/s]
39%|โโโโ | 615/1565 [10:04<15:40, 1.01it/s]
39%|โโโโ | 616/1565 [10:05<15:35, 1.01it/s]
39%|โโโโ | 617/1565 [10:06<15:31, 1.02it/s]
39%|โโโโ | 618/1565 [10:07<15:28, 1.02it/s]
40%|โโโโ | 619/1565 [10:08<15:24, 1.02it/s]
40%|โโโโ | 620/1565 [10:09<15:22, 1.02it/s]
40%|โโโโ | 621/1565 [10:10<15:20, 1.03it/s]
40%|โโโโ | 622/1565 [10:11<15:20, 1.02it/s]
40%|โโโโ | 623/1565 [10:12<15:20, 1.02it/s]
40%|โโโโ | 624/1565 [10:13<15:18, 1.02it/s]
40%|โโโโ | 625/1565 [10:14<15:06, 1.04it/s]
40%|โโโโ | 626/1565 [10:14<12:51, 1.22it/s]
40%|โโโโ | 627/1565 [10:16<13:57, 1.12it/s]
40%|โโโโ | 628/1565 [10:17<14:19, 1.09it/s]
40%|โโโโ | 629/1565 [10:17<14:32, 1.07it/s]
40%|โโโโ | 630/1565 [10:18<14:41, 1.06it/s]
40%|โโโโ | 631/1565 [10:19<14:49, 1.05it/s]
40%|โโโโ | 632/1565 [10:20<14:52, 1.05it/s]
40%|โโโโ | 633/1565 [10:21<14:55, 1.04it/s]
41%|โโโโ | 634/1565 [10:22<14:57, 1.04it/s]
41%|โโโโ | 635/1565 [10:23<14:59, 1.03it/s]
41%|โโโโ | 636/1565 [10:24<14:58, 1.03it/s]
41%|โโโโ | 637/1565 [10:25<14:57, 1.03it/s]
41%|โโโโ | 638/1565 [10:26<14:57, 1.03it/s]
41%|โโโโ | 639/1565 [10:27<14:56, 1.03it/s]
41%|โโโโ | 640/1565 [10:28<14:54, 1.03it/s]
41%|โโโโ | 641/1565 [10:29<14:54, 1.03it/s]
41%|โโโโ | 642/1565 [10:30<14:55, 1.03it/s]
41%|โโโโ | 643/1565 [10:31<14:55, 1.03it/s]
41%|โโโโ | 644/1565 [10:32<14:56, 1.03it/s]
41%|โโโโ | 645/1565 [10:33<14:54, 1.03it/s]
41%|โโโโโ | 646/1565 [10:34<14:52, 1.03it/s]
41%|โโโโโ | 647/1565 [10:35<14:50, 1.03it/s]
41%|โโโโโ | 648/1565 [10:36<14:49, 1.03it/s]
41%|โโโโโ | 649/1565 [10:37<14:49, 1.03it/s]
42%|โโโโโ | 650/1565 [10:38<14:48, 1.03it/s]
42%|โโโโโ | 651/1565 [10:39<14:48, 1.03it/s]
42%|โโโโโ | 652/1565 [10:40<14:46, 1.03it/s]
42%|โโโโโ | 653/1565 [10:41<14:46, 1.03it/s]
42%|โโโโโ | 654/1565 [10:42<14:44, 1.03it/s]
42%|โโโโโ | 655/1565 [10:43<14:43, 1.03it/s]
42%|โโโโโ | 656/1565 [10:44<14:41, 1.03it/s]
42%|โโโโโ | 657/1565 [10:45<14:43, 1.03it/s]
42%|โโโโโ | 658/1565 [10:46<15:46, 1.04s/it]
42%|โโโโโ | 659/1565 [10:47<15:27, 1.02s/it]
42%|โโโโโ | 660/1565 [10:48<15:12, 1.01s/it]
42%|โโโโโ | 661/1565 [10:49<15:03, 1.00it/s]
42%|โโโโโ | 662/1565 [10:50<14:55, 1.01it/s]
42%|โโโโโ | 663/1565 [10:51<14:49, 1.01it/s]
42%|โโโโโ | 664/1565 [10:52<14:44, 1.02it/s]
42%|โโโโโ | 665/1565 [10:53<14:40, 1.02it/s]
43%|โโโโโ | 666/1565 [10:54<14:38, 1.02it/s]
43%|โโโโโ | 667/1565 [10:55<14:34, 1.03it/s]
43%|โโโโโ | 668/1565 [10:56<14:33, 1.03it/s]
43%|โโโโโ | 669/1565 [10:57<14:32, 1.03it/s]
43%|โโโโโ | 670/1565 [10:58<14:31, 1.03it/s]
43%|โโโโโ | 671/1565 [10:59<14:31, 1.03it/s]
43%|โโโโโ | 672/1565 [10:59<14:29, 1.03it/s]
43%|โโโโโ | 673/1565 [11:00<14:28, 1.03it/s]
43%|โโโโโ | 674/1565 [11:01<14:28, 1.03it/s]
43%|โโโโโ | 675/1565 [11:02<14:26, 1.03it/s]
43%|โโโโโ | 676/1565 [11:03<14:24, 1.03it/s]
43%|โโโโโ | 677/1565 [11:04<14:23, 1.03it/s]
43%|โโโโโ | 678/1565 [11:05<14:23, 1.03it/s]
43%|โโโโโ | 679/1565 [11:06<14:21, 1.03it/s]
43%|โโโโโ | 680/1565 [11:07<14:20, 1.03it/s]
44%|โโโโโ | 681/1565 [11:08<14:20, 1.03it/s]
44%|โโโโโ | 682/1565 [11:09<14:19, 1.03it/s]
44%|โโโโโ | 683/1565 [11:10<14:18, 1.03it/s]
44%|โโโโโ | 684/1565 [11:11<14:19, 1.03it/s]
44%|โโโโโ | 685/1565 [11:12<14:18, 1.03it/s]
44%|โโโโโ | 686/1565 [11:13<14:16, 1.03it/s]
44%|โโโโโ | 687/1565 [11:14<14:15, 1.03it/s]
44%|โโโโโ | 688/1565 [11:15<14:14, 1.03it/s]
44%|โโโโโ | 689/1565 [11:16<14:14, 1.03it/s]
44%|โโโโโ | 690/1565 [11:17<14:13, 1.03it/s]
44%|โโโโโ | 691/1565 [11:18<14:10, 1.03it/s]
44%|โโโโโ | 692/1565 [11:19<14:07, 1.03it/s]
44%|โโโโโ | 693/1565 [11:20<14:06, 1.03it/s]
44%|โโโโโ | 694/1565 [11:21<14:05, 1.03it/s]
44%|โโโโโ | 695/1565 [11:22<14:03, 1.03it/s]
44%|โโโโโ | 696/1565 [11:23<14:01, 1.03it/s]
45%|โโโโโ | 697/1565 [11:24<14:00, 1.03it/s]
45%|โโโโโ | 698/1565 [11:25<14:01, 1.03it/s]
45%|โโโโโ | 699/1565 [11:26<14:00, 1.03it/s]
45%|โโโโโ | 700/1565 [11:27<13:59, 1.03it/s]
45%|โโโโโ | 701/1565 [11:28<13:58, 1.03it/s]
45%|โโโโโ | 702/1565 [11:29<13:58, 1.03it/s]
45%|โโโโโ | 703/1565 [11:30<13:56, 1.03it/s]
45%|โโโโโ | 704/1565 [11:31<13:55, 1.03it/s]
45%|โโโโโ | 705/1565 [11:32<14:54, 1.04s/it]
45%|โโโโโ | 706/1565 [11:33<14:34, 1.02s/it]
45%|โโโโโ | 707/1565 [11:34<14:21, 1.00s/it]
45%|โโโโโ | 708/1565 [11:35<14:13, 1.00it/s]
45%|โโโโโ | 709/1565 [11:36<14:07, 1.01it/s]
45%|โโโโโ | 710/1565 [11:37<14:01, 1.02it/s]
45%|โโโโโ | 711/1565 [11:38<13:56, 1.02it/s]
45%|โโโโโ | 712/1565 [11:39<13:52, 1.02it/s]
46%|โโโโโ | 713/1565 [11:40<13:50, 1.03it/s]
46%|โโโโโ | 714/1565 [11:41<13:49, 1.03it/s]
46%|โโโโโ | 715/1565 [11:42<13:47, 1.03it/s]
46%|โโโโโ | 716/1565 [11:42<13:46, 1.03it/s]
46%|โโโโโ | 717/1565 [11:43<13:45, 1.03it/s]
46%|โโโโโ | 718/1565 [11:44<13:46, 1.02it/s]
46%|โโโโโ | 719/1565 [11:45<13:45, 1.02it/s]
46%|โโโโโ | 720/1565 [11:46<13:42, 1.03it/s]
46%|โโโโโ | 721/1565 [11:47<13:39, 1.03it/s]
46%|โโโโโ | 722/1565 [11:48<13:39, 1.03it/s]
46%|โโโโโ | 723/1565 [11:49<13:37, 1.03it/s]
46%|โโโโโ | 724/1565 [11:50<13:36, 1.03it/s]
46%|โโโโโ | 725/1565 [11:51<13:34, 1.03it/s]
46%|โโโโโ | 726/1565 [11:52<13:35, 1.03it/s]
46%|โโโโโ | 727/1565 [11:53<13:33, 1.03it/s]
47%|โโโโโ | 728/1565 [11:54<13:32, 1.03it/s]
47%|โโโโโ | 729/1565 [11:55<13:30, 1.03it/s]
47%|โโโโโ | 730/1565 [11:56<13:30, 1.03it/s]
47%|โโโโโ | 731/1565 [11:57<13:28, 1.03it/s]
47%|โโโโโ | 732/1565 [11:58<13:27, 1.03it/s]
47%|โโโโโ | 733/1565 [11:59<13:28, 1.03it/s]
47%|โโโโโ | 734/1565 [12:00<13:28, 1.03it/s]
47%|โโโโโ | 735/1565 [12:01<13:27, 1.03it/s]
47%|โโโโโ | 736/1565 [12:02<13:26, 1.03it/s]
47%|โโโโโ | 737/1565 [12:03<13:26, 1.03it/s]
47%|โโโโโ | 738/1565 [12:04<13:26, 1.03it/s]
47%|โโโโโ | 739/1565 [12:05<13:25, 1.03it/s]
47%|โโโโโ | 740/1565 [12:06<13:23, 1.03it/s]
47%|โโโโโ | 741/1565 [12:07<13:22, 1.03it/s]
47%|โโโโโ | 742/1565 [12:08<13:20, 1.03it/s]
47%|โโโโโ | 743/1565 [12:09<13:19, 1.03it/s]
48%|โโโโโ | 744/1565 [12:10<13:17, 1.03it/s]
48%|โโโโโ | 745/1565 [12:11<13:15, 1.03it/s]
48%|โโโโโ | 746/1565 [12:12<13:14, 1.03it/s]
48%|โโโโโ | 747/1565 [12:13<13:14, 1.03it/s]
48%|โโโโโ | 748/1565 [12:14<13:14, 1.03it/s]
48%|โโโโโ | 749/1565 [12:15<13:14, 1.03it/s]
48%|โโโโโ | 750/1565 [12:16<13:14, 1.03it/s]
48%|โโโโโ | 751/1565 [12:17<13:13, 1.03it/s]
48%|โโโโโ | 752/1565 [12:18<14:07, 1.04s/it]
48%|โโโโโ | 753/1565 [12:19<13:49, 1.02s/it]
48%|โโโโโ | 754/1565 [12:20<13:37, 1.01s/it]
48%|โโโโโ | 755/1565 [12:21<13:27, 1.00it/s]
48%|โโโโโ | 756/1565 [12:22<13:21, 1.01it/s]
48%|โโโโโ | 757/1565 [12:23<13:15, 1.02it/s]
48%|โโโโโ | 758/1565 [12:24<13:12, 1.02it/s]
48%|โโโโโ | 759/1565 [12:25<13:09, 1.02it/s]
49%|โโโโโ | 760/1565 [12:26<13:06, 1.02it/s]
49%|โโโโโ | 761/1565 [12:26<13:03, 1.03it/s]
49%|โโโโโ | 762/1565 [12:27<13:02, 1.03it/s]
49%|โโโโโ | 763/1565 [12:28<13:00, 1.03it/s]
49%|โโโโโ | 764/1565 [12:29<12:58, 1.03it/s]
49%|โโโโโ | 765/1565 [12:30<12:57, 1.03it/s]
49%|โโโโโ | 766/1565 [12:31<12:57, 1.03it/s]
49%|โโโโโ | 767/1565 [12:32<12:56, 1.03it/s]
49%|โโโโโ | 768/1565 [12:33<12:55, 1.03it/s]
49%|โโโโโ | 769/1565 [12:34<12:54, 1.03it/s]
49%|โโโโโ | 770/1565 [12:35<12:54, 1.03it/s]
49%|โโโโโ | 771/1565 [12:36<12:53, 1.03it/s]
49%|โโโโโ | 772/1565 [12:37<12:51, 1.03it/s]
49%|โโโโโ | 773/1565 [12:38<12:49, 1.03it/s]
49%|โโโโโ | 774/1565 [12:39<12:48, 1.03it/s]
50%|โโโโโ | 775/1565 [12:40<12:48, 1.03it/s]
50%|โโโโโ | 776/1565 [12:41<12:47, 1.03it/s]
50%|โโโโโ | 777/1565 [12:42<12:48, 1.03it/s]
50%|โโโโโ | 778/1565 [12:43<12:46, 1.03it/s]
50%|โโโโโ | 779/1565 [12:44<12:44, 1.03it/s]
50%|โโโโโ | 780/1565 [12:45<12:42, 1.03it/s]
50%|โโโโโ | 781/1565 [12:46<12:41, 1.03it/s]
50%|โโโโโ | 782/1565 [12:47<12:40, 1.03it/s]
50%|โโโโโ | 783/1565 [12:48<12:39, 1.03it/s]
50%|โโโโโ | 784/1565 [12:49<12:37, 1.03it/s]
50%|โโโโโ | 785/1565 [12:50<12:37, 1.03it/s]
50%|โโโโโ | 786/1565 [12:51<12:38, 1.03it/s]
50%|โโโโโ | 787/1565 [12:52<12:37, 1.03it/s]
50%|โโโโโ | 788/1565 [12:53<12:37, 1.03it/s]
50%|โโโโโ | 789/1565 [12:54<12:35, 1.03it/s]
50%|โโโโโ | 790/1565 [12:55<12:33, 1.03it/s]
51%|โโโโโ | 791/1565 [12:56<12:32, 1.03it/s]
51%|โโโโโ | 792/1565 [12:57<12:32, 1.03it/s]
51%|โโโโโ | 793/1565 [12:58<12:31, 1.03it/s]
51%|โโโโโ | 794/1565 [12:59<12:30, 1.03it/s]
51%|โโโโโ | 795/1565 [13:00<12:28, 1.03it/s]
51%|โโโโโ | 796/1565 [13:01<12:28, 1.03it/s]
51%|โโโโโ | 797/1565 [13:02<12:25, 1.03it/s]
51%|โโโโโ | 798/1565 [13:02<12:24, 1.03it/s]
51%|โโโโโ | 799/1565 [13:04<13:18, 1.04s/it]
51%|โโโโโ | 800/1565 [13:05<13:02, 1.02s/it]
51%|โโโโโ | 801/1565 [13:06<12:50, 1.01s/it]
51%|โโโโโ | 802/1565 [13:07<12:41, 1.00it/s]
51%|โโโโโโ | 803/1565 [13:08<12:32, 1.01it/s]
51%|โโโโโโ | 804/1565 [13:09<12:26, 1.02it/s]
51%|โโโโโโ | 805/1565 [13:10<12:23, 1.02it/s]
52%|โโโโโโ | 806/1565 [13:10<12:21, 1.02it/s]
52%|โโโโโโ | 807/1565 [13:11<12:19, 1.02it/s]
52%|โโโโโโ | 808/1565 [13:12<12:18, 1.03it/s]
52%|โโโโโโ | 809/1565 [13:13<12:17, 1.02it/s]
52%|โโโโโโ | 810/1565 [13:14<12:16, 1.03it/s]
52%|โโโโโโ | 811/1565 [13:15<12:15, 1.03it/s]
52%|โโโโโโ | 812/1565 [13:16<12:14, 1.03it/s]
52%|โโโโโโ | 813/1565 [13:17<12:11, 1.03it/s]
52%|โโโโโโ | 814/1565 [13:18<12:09, 1.03it/s]
52%|โโโโโโ | 815/1565 [13:19<12:07, 1.03it/s]
52%|โโโโโโ | 816/1565 [13:20<12:05, 1.03it/s]
52%|โโโโโโ | 817/1565 [13:21<12:06, 1.03it/s]
52%|โโโโโโ | 818/1565 [13:22<12:06, 1.03it/s]
52%|โโโโโโ | 819/1565 [13:23<12:05, 1.03it/s]
52%|โโโโโโ | 820/1565 [13:24<12:05, 1.03it/s]
52%|โโโโโโ | 821/1565 [13:25<12:05, 1.03it/s]
53%|โโโโโโ | 822/1565 [13:26<12:02, 1.03it/s]
53%|โโโโโโ | 823/1565 [13:27<12:02, 1.03it/s]
53%|โโโโโโ | 824/1565 [13:28<12:00, 1.03it/s]
53%|โโโโโโ | 825/1565 [13:29<11:59, 1.03it/s]
53%|โโโโโโ | 826/1565 [13:30<11:59, 1.03it/s]
53%|โโโโโโ | 827/1565 [13:31<11:58, 1.03it/s]
53%|โโโโโโ | 828/1565 [13:32<11:57, 1.03it/s]
53%|โโโโโโ | 829/1565 [13:33<11:56, 1.03it/s]
53%|โโโโโโ | 830/1565 [13:34<11:55, 1.03it/s]
53%|โโโโโโ | 831/1565 [13:35<11:55, 1.03it/s]
53%|โโโโโโ | 832/1565 [13:36<11:54, 1.03it/s]
53%|โโโโโโ | 833/1565 [13:37<11:52, 1.03it/s]
53%|โโโโโโ | 834/1565 [13:38<11:53, 1.02it/s]
53%|โโโโโโ | 835/1565 [13:39<11:51, 1.03it/s]
53%|โโโโโโ | 836/1565 [13:40<11:50, 1.03it/s]
53%|โโโโโโ | 837/1565 [13:41<11:48, 1.03it/s]
54%|โโโโโโ | 838/1565 [13:42<11:47, 1.03it/s]
54%|โโโโโโ | 839/1565 [13:43<11:46, 1.03it/s]
54%|โโโโโโ | 840/1565 [13:44<11:46, 1.03it/s]
54%|โโโโโโ | 841/1565 [13:45<11:45, 1.03it/s]
54%|โโโโโโ | 842/1565 [13:46<11:44, 1.03it/s]
54%|โโโโโโ | 843/1565 [13:47<11:43, 1.03it/s]
54%|โโโโโโ | 844/1565 [13:47<11:41, 1.03it/s]
54%|โโโโโโ | 845/1565 [13:48<11:40, 1.03it/s]
54%|โโโโโโ | 846/1565 [13:50<12:28, 1.04s/it]
54%|โโโโโโ | 847/1565 [13:51<12:13, 1.02s/it]
54%|โโโโโโ | 848/1565 [13:52<12:02, 1.01s/it]
54%|โโโโโโ | 849/1565 [13:53<11:54, 1.00it/s]
54%|โโโโโโ | 850/1565 [13:54<11:47, 1.01it/s]
54%|โโโโโโ | 851/1565 [13:55<11:43, 1.01it/s]
54%|โโโโโโ | 852/1565 [13:55<11:39, 1.02it/s]
55%|โโโโโโ | 853/1565 [13:56<11:36, 1.02it/s]
55%|โโโโโโ | 854/1565 [13:57<11:33, 1.02it/s]
55%|โโโโโโ | 855/1565 [13:58<11:32, 1.03it/s]
55%|โโโโโโ | 856/1565 [13:59<11:31, 1.03it/s]
55%|โโโโโโ | 857/1565 [14:00<11:30, 1.03it/s]
55%|โโโโโโ | 858/1565 [14:01<11:29, 1.03it/s]
55%|โโโโโโ | 859/1565 [14:02<11:27, 1.03it/s]
55%|โโโโโโ | 860/1565 [14:03<11:26, 1.03it/s]
55%|โโโโโโ | 861/1565 [14:04<11:25, 1.03it/s]
55%|โโโโโโ | 862/1565 [14:05<11:24, 1.03it/s]
55%|โโโโโโ | 863/1565 [14:06<11:23, 1.03it/s]
55%|โโโโโโ | 864/1565 [14:07<11:22, 1.03it/s]
55%|โโโโโโ | 865/1565 [14:08<11:22, 1.03it/s]
55%|โโโโโโ | 866/1565 [14:09<11:21, 1.03it/s]
55%|โโโโโโ | 867/1565 [14:10<11:20, 1.03it/s]
55%|โโโโโโ | 868/1565 [14:11<11:18, 1.03it/s]
56%|โโโโโโ | 869/1565 [14:12<11:17, 1.03it/s]
56%|โโโโโโ | 870/1565 [14:13<11:16, 1.03it/s]
56%|โโโโโโ | 871/1565 [14:14<11:14, 1.03it/s]
56%|โโโโโโ | 872/1565 [14:15<11:14, 1.03it/s]
56%|โโโโโโ | 873/1565 [14:16<11:13, 1.03it/s]
56%|โโโโโโ | 874/1565 [14:17<11:13, 1.03it/s]
56%|โโโโโโ | 875/1565 [14:18<11:12, 1.03it/s]
56%|โโโโโโ | 876/1565 [14:19<11:11, 1.03it/s]
56%|โโโโโโ | 877/1565 [14:20<11:10, 1.03it/s]
56%|โโโโโโ | 878/1565 [14:21<11:08, 1.03it/s]
56%|โโโโโโ | 879/1565 [14:22<11:08, 1.03it/s]
56%|โโโโโโ | 880/1565 [14:23<11:07, 1.03it/s]
56%|โโโโโโ | 881/1565 [14:24<11:06, 1.03it/s]
56%|โโโโโโ | 882/1565 [14:25<11:05, 1.03it/s]
56%|โโโโโโ | 883/1565 [14:26<11:04, 1.03it/s]
56%|โโโโโโ | 884/1565 [14:27<11:03, 1.03it/s]
57%|โโโโโโ | 885/1565 [14:28<11:01, 1.03it/s]
57%|โโโโโโ | 886/1565 [14:29<10:59, 1.03it/s]
57%|โโโโโโ | 887/1565 [14:30<10:57, 1.03it/s]
57%|โโโโโโ | 888/1565 [14:31<10:56, 1.03it/s]
57%|โโโโโโ | 889/1565 [14:31<10:56, 1.03it/s]
57%|โโโโโโ | 890/1565 [14:32<10:56, 1.03it/s]
57%|โโโโโโ | 891/1565 [14:33<10:56, 1.03it/s]
57%|โโโโโโ | 892/1565 [14:34<10:56, 1.03it/s]
57%|โโโโโโ | 893/1565 [14:36<11:41, 1.04s/it]
57%|โโโโโโ | 894/1565 [14:37<11:27, 1.02s/it]
57%|โโโโโโ | 895/1565 [14:38<11:14, 1.01s/it]
57%|โโโโโโ | 896/1565 [14:39<11:06, 1.00it/s]
57%|โโโโโโ | 897/1565 [14:40<11:01, 1.01it/s]
57%|โโโโโโ | 898/1565 [14:40<10:56, 1.02it/s]
57%|โโโโโโ | 899/1565 [14:41<10:53, 1.02it/s]
58%|โโโโโโ | 900/1565 [14:42<10:50, 1.02it/s]
58%|โโโโโโ | 901/1565 [14:43<10:47, 1.02it/s]
58%|โโโโโโ | 902/1565 [14:44<10:45, 1.03it/s]
58%|โโโโโโ | 903/1565 [14:45<10:43, 1.03it/s]
58%|โโโโโโ | 904/1565 [14:46<10:42, 1.03it/s]
58%|โโโโโโ | 905/1565 [14:47<10:39, 1.03it/s]
58%|โโโโโโ | 906/1565 [14:48<10:38, 1.03it/s]
58%|โโโโโโ | 907/1565 [14:49<10:37, 1.03it/s]
58%|โโโโโโ | 908/1565 [14:50<10:37, 1.03it/s]
58%|โโโโโโ | 909/1565 [14:51<10:37, 1.03it/s]
58%|โโโโโโ | 910/1565 [14:52<10:36, 1.03it/s]
58%|โโโโโโ | 911/1565 [14:53<10:35, 1.03it/s]
58%|โโโโโโ | 912/1565 [14:54<10:34, 1.03it/s]
58%|โโโโโโ | 913/1565 [14:55<10:32, 1.03it/s]
58%|โโโโโโ | 914/1565 [14:56<10:31, 1.03it/s]
58%|โโโโโโ | 915/1565 [14:57<10:31, 1.03it/s]
59%|โโโโโโ | 916/1565 [14:58<10:30, 1.03it/s]
59%|โโโโโโ | 917/1565 [14:59<10:29, 1.03it/s]
59%|โโโโโโ | 918/1565 [15:00<10:27, 1.03it/s]
59%|โโโโโโ | 919/1565 [15:01<10:27, 1.03it/s]
59%|โโโโโโ | 920/1565 [15:02<10:25, 1.03it/s]
59%|โโโโโโ | 921/1565 [15:03<10:24, 1.03it/s]
59%|โโโโโโ | 922/1565 [15:04<10:24, 1.03it/s]
59%|โโโโโโ | 923/1565 [15:05<10:23, 1.03it/s]
59%|โโโโโโ | 924/1565 [15:06<10:23, 1.03it/s]
59%|โโโโโโ | 925/1565 [15:07<10:23, 1.03it/s]
59%|โโโโโโ | 926/1565 [15:08<10:21, 1.03it/s]
59%|โโโโโโ | 927/1565 [15:09<10:21, 1.03it/s]
59%|โโโโโโ | 928/1565 [15:10<10:20, 1.03it/s]
59%|โโโโโโ | 929/1565 [15:11<10:20, 1.02it/s]
59%|โโโโโโ | 930/1565 [15:12<10:18, 1.03it/s]
59%|โโโโโโ | 931/1565 [15:13<10:16, 1.03it/s]
60%|โโโโโโ | 932/1565 [15:14<10:16, 1.03it/s]
60%|โโโโโโ | 933/1565 [15:15<10:15, 1.03it/s]
60%|โโโโโโ | 934/1565 [15:15<10:15, 1.03it/s]
60%|โโโโโโ | 935/1565 [15:16<10:13, 1.03it/s]
60%|โโโโโโ | 936/1565 [15:17<10:12, 1.03it/s]
60%|โโโโโโ | 937/1565 [15:18<10:11, 1.03it/s]
60%|โโโโโโ | 938/1565 [15:19<10:04, 1.04it/s]
60%|โโโโโโ | 939/1565 [15:20<08:33, 1.22it/s]
60%|โโโโโโ | 940/1565 [15:21<10:00, 1.04it/s]
60%|โโโโโโ | 941/1565 [15:22<10:02, 1.04it/s]
60%|โโโโโโ | 942/1565 [15:23<10:03, 1.03it/s]
60%|โโโโโโ | 943/1565 [15:24<10:02, 1.03it/s]
60%|โโโโโโ | 944/1565 [15:25<10:02, 1.03it/s]
60%|โโโโโโ | 945/1565 [15:26<10:01, 1.03it/s]
60%|โโโโโโ | 946/1565 [15:27<10:01, 1.03it/s]
61%|โโโโโโ | 947/1565 [15:28<09:59, 1.03it/s]
61%|โโโโโโ | 948/1565 [15:29<09:59, 1.03it/s]
61%|โโโโโโ | 949/1565 [15:30<09:58, 1.03it/s]
61%|โโโโโโ | 950/1565 [15:31<09:57, 1.03it/s]
61%|โโโโโโ | 951/1565 [15:32<09:57, 1.03it/s]
61%|โโโโโโ | 952/1565 [15:33<09:56, 1.03it/s]
61%|โโโโโโ | 953/1565 [15:34<09:55, 1.03it/s]
61%|โโโโโโ | 954/1565 [15:35<09:54, 1.03it/s]
61%|โโโโโโ | 955/1565 [15:36<09:53, 1.03it/s]
61%|โโโโโโ | 956/1565 [15:37<09:52, 1.03it/s]
61%|โโโโโโ | 957/1565 [15:38<09:51, 1.03it/s]
61%|โโโโโโ | 958/1565 [15:39<09:50, 1.03it/s]
61%|โโโโโโโ | 959/1565 [15:40<09:49, 1.03it/s]
61%|โโโโโโโ | 960/1565 [15:41<09:48, 1.03it/s]
61%|โโโโโโโ | 961/1565 [15:42<09:48, 1.03it/s]
61%|โโโโโโโ | 962/1565 [15:43<09:46, 1.03it/s]
62%|โโโโโโโ | 963/1565 [15:43<09:45, 1.03it/s]
62%|โโโโโโโ | 964/1565 [15:44<09:44, 1.03it/s]
62%|โโโโโโโ | 965/1565 [15:45<09:42, 1.03it/s]
62%|โโโโโโโ | 966/1565 [15:46<09:40, 1.03it/s]
62%|โโโโโโโ | 967/1565 [15:47<09:38, 1.03it/s]
62%|โโโโโโโ | 968/1565 [15:48<09:37, 1.03it/s]
62%|โโโโโโโ | 969/1565 [15:49<09:36, 1.03it/s]
62%|โโโโโโโ | 970/1565 [15:50<09:36, 1.03it/s]
62%|โโโโโโโ | 971/1565 [15:51<09:36, 1.03it/s]
62%|โโโโโโโ | 972/1565 [15:52<09:34, 1.03it/s]
62%|โโโโโโโ | 973/1565 [15:53<09:34, 1.03it/s]
62%|โโโโโโโ | 974/1565 [15:54<09:34, 1.03it/s]
62%|โโโโโโโ | 975/1565 [15:55<09:32, 1.03it/s]
62%|โโโโโโโ | 976/1565 [15:56<09:30, 1.03it/s]
62%|โโโโโโโ | 977/1565 [15:57<09:30, 1.03it/s]
62%|โโโโโโโ | 978/1565 [15:58<09:30, 1.03it/s]
63%|โโโโโโโ | 979/1565 [15:59<09:29, 1.03it/s]
63%|โโโโโโโ | 980/1565 [16:00<09:27, 1.03it/s]
63%|โโโโโโโ | 981/1565 [16:01<09:26, 1.03it/s]
63%|โโโโโโโ | 982/1565 [16:02<09:25, 1.03it/s]
63%|โโโโโโโ | 983/1565 [16:03<09:25, 1.03it/s]
63%|โโโโโโโ | 984/1565 [16:04<09:24, 1.03it/s]
63%|โโโโโโโ | 985/1565 [16:05<09:23, 1.03it/s]
63%|โโโโโโโ | 986/1565 [16:06<09:22, 1.03it/s]
63%|โโโโโโโ | 987/1565 [16:07<10:01, 1.04s/it]
63%|โโโโโโโ | 988/1565 [16:08<09:48, 1.02s/it]
63%|โโโโโโโ | 989/1565 [16:09<09:40, 1.01s/it]
63%|โโโโโโโ | 990/1565 [16:10<09:34, 1.00it/s]
63%|โโโโโโโ | 991/1565 [16:11<09:29, 1.01it/s]
63%|โโโโโโโ | 992/1565 [16:12<09:24, 1.01it/s]
63%|โโโโโโโ | 993/1565 [16:13<09:20, 1.02it/s]
64%|โโโโโโโ | 994/1565 [16:14<09:18, 1.02it/s]
64%|โโโโโโโ | 995/1565 [16:15<09:16, 1.02it/s]
64%|โโโโโโโ | 996/1565 [16:16<09:15, 1.02it/s]
64%|โโโโโโโ | 997/1565 [16:17<09:13, 1.03it/s]
64%|โโโโโโโ | 998/1565 [16:18<09:11, 1.03it/s]
64%|โโโโโโโ | 999/1565 [16:19<09:11, 1.03it/s]
64%|โโโโโโโ | 1000/1565 [16:20<09:10, 1.03it/s]
64%|โโโโโโโ | 1000/1565 [16:20<09:10, 1.03it/s]{'loss': 0.5945, 'grad_norm': 5.812021732330322, 'learning_rate': 1.9007625957379082e-06, 'epoch': 3.19}
0%| | 0/79 [00:00<?, ?it/s][A
3%|โ | 2/79 [00:00<00:07, 10.44it/s][A
5%|โ | 4/79 [00:00<00:11, 6.57it/s][A
6%|โ | 5/79 [00:00<00:12, 6.12it/s][A
8%|โ | 6/79 [00:00<00:12, 5.83it/s][A
9%|โ | 7/79 [00:01<00:12, 5.65it/s][A
10%|โ | 8/79 [00:01<00:12, 5.53it/s][A
11%|โโ | 9/79 [00:01<00:12, 5.44it/s][A
13%|โโ | 10/79 [00:01<00:12, 5.37it/s][A
14%|โโ | 11/79 [00:01<00:12, 5.33it/s][A
15%|โโ | 12/79 [00:02<00:12, 5.29it/s][A
16%|โโ | 13/79 [00:02<00:12, 5.28it/s][A
18%|โโ | 14/79 [00:02<00:12, 5.26it/s][A
19%|โโ | 15/79 [00:02<00:12, 5.25it/s][A
20%|โโ | 16/79 [00:02<00:12, 5.24it/s][A
22%|โโโ | 17/79 [00:03<00:11, 5.24it/s][A
23%|โโโ | 18/79 [00:03<00:11, 5.22it/s][A
24%|โโโ | 19/79 [00:03<00:11, 5.23it/s][A
25%|โโโ | 20/79 [00:03<00:11, 5.22it/s][A
27%|โโโ | 21/79 [00:03<00:11, 5.20it/s][A
28%|โโโ | 22/79 [00:04<00:10, 5.22it/s][A
29%|โโโ | 23/79 [00:04<00:10, 5.24it/s][A
30%|โโโ | 24/79 [00:04<00:10, 5.22it/s][A
32%|โโโโ | 25/79 [00:04<00:10, 5.22it/s][A
33%|โโโโ | 26/79 [00:04<00:10, 5.21it/s][A
34%|โโโโ | 27/79 [00:04<00:10, 5.20it/s][A
35%|โโโโ | 28/79 [00:05<00:09, 5.19it/s][A
37%|โโโโ | 29/79 [00:05<00:09, 5.19it/s][A
38%|โโโโ | 30/79 [00:05<00:09, 5.22it/s][A
39%|โโโโ | 31/79 [00:05<00:09, 5.24it/s][A
41%|โโโโ | 32/79 [00:05<00:08, 5.24it/s][A
42%|โโโโโ | 33/79 [00:06<00:08, 5.24it/s][A
43%|โโโโโ | 34/79 [00:06<00:08, 5.25it/s][A
44%|โโโโโ | 35/79 [00:06<00:08, 5.25it/s][A
46%|โโโโโ | 36/79 [00:06<00:08, 5.24it/s][A
47%|โโโโโ | 37/79 [00:06<00:07, 5.25it/s][A
48%|โโโโโ | 38/79 [00:07<00:07, 5.24it/s][A
49%|โโโโโ | 39/79 [00:07<00:07, 5.23it/s][A
51%|โโโโโ | 40/79 [00:07<00:10, 3.85it/s][A
52%|โโโโโโ | 41/79 [00:07<00:09, 4.18it/s][A
53%|โโโโโโ | 42/79 [00:08<00:08, 4.45it/s][A
54%|โโโโโโ | 43/79 [00:08<00:07, 4.67it/s][A
56%|โโโโโโ | 44/79 [00:08<00:07, 4.83it/s][A
57%|โโโโโโ | 45/79 [00:08<00:06, 4.94it/s][A
58%|โโโโโโ | 46/79 [00:08<00:06, 5.02it/s][A
59%|โโโโโโ | 47/79 [00:09<00:06, 5.09it/s][A
61%|โโโโโโ | 48/79 [00:09<00:06, 5.13it/s][A
62%|โโโโโโโ | 49/79 [00:09<00:05, 5.18it/s][A
63%|โโโโโโโ | 50/79 [00:09<00:05, 5.20it/s][A
65%|โโโโโโโ | 51/79 [00:09<00:05, 5.19it/s][A
66%|โโโโโโโ | 52/79 [00:09<00:05, 5.19it/s][A
67%|โโโโโโโ | 53/79 [00:10<00:05, 5.18it/s][A
68%|โโโโโโโ | 54/79 [00:10<00:04, 5.19it/s][A
70%|โโโโโโโ | 55/79 [00:10<00:04, 5.18it/s][A
71%|โโโโโโโ | 56/79 [00:10<00:04, 5.21it/s][A
72%|โโโโโโโโ | 57/79 [00:10<00:04, 5.22it/s][A
73%|โโโโโโโโ | 58/79 [00:11<00:04, 5.23it/s][A
75%|โโโโโโโโ | 59/79 [00:11<00:03, 5.23it/s][A
76%|โโโโโโโโ | 60/79 [00:11<00:03, 5.23it/s][A
77%|โโโโโโโโ | 61/79 [00:11<00:03, 5.24it/s][A
78%|โโโโโโโโ | 62/79 [00:11<00:03, 5.25it/s][A
80%|โโโโโโโโ | 63/79 [00:12<00:03, 5.25it/s][A
81%|โโโโโโโโ | 64/79 [00:12<00:02, 5.25it/s][A
82%|โโโโโโโโโ | 65/79 [00:12<00:02, 5.22it/s][A
84%|โโโโโโโโโ | 66/79 [00:12<00:02, 5.22it/s][A
85%|โโโโโโโโโ | 67/79 [00:12<00:02, 5.21it/s][A
86%|โโโโโโโโโ | 68/79 [00:13<00:02, 5.20it/s][A
87%|โโโโโโโโโ | 69/79 [00:13<00:01, 5.20it/s][A
89%|โโโโโโโโโ | 70/79 [00:13<00:01, 5.20it/s][A
90%|โโโโโโโโโ | 71/79 [00:13<00:01, 5.20it/s][A
91%|โโโโโโโโโ | 72/79 [00:13<00:01, 5.20it/s][A
92%|โโโโโโโโโโ| 73/79 [00:14<00:01, 5.21it/s][A
94%|โโโโโโโโโโ| 74/79 [00:14<00:00, 5.22it/s][A
95%|โโโโโโโโโโ| 75/79 [00:14<00:00, 5.23it/s][A
96%|โโโโโโโโโโ| 76/79 [00:14<00:00, 5.23it/s][A
97%|โโโโโโโโโโ| 77/79 [00:14<00:00, 5.24it/s][A
99%|โโโโโโโโโโ| 78/79 [00:14<00:00, 5.50it/s][A[W 2025-11-26 12:22:27,446] Trial 0 failed with parameters: {'learning_rate': 5.2556421595933325e-06, 'num_train_epochs': 5, 'seed': 12, 'per_device_train_batch_size': 32} because of the following error: TypeError("'float' object does not support item assignment").
Traceback (most recent call last):
File "/tmp/conda/wzh/lib/python3.11/site-packages/optuna/study/_optimize.py", line 205, in _run_trial
value_or_values = func(trial)
^^^^^^^^^^^
File "/tmp/conda/wzh/lib/python3.11/site-packages/transformers/integrations/integration_utils.py", line 277, in _objective
trainer.train(resume_from_checkpoint=checkpoint, trial=trial)
File "/tmp/conda/wzh/lib/python3.11/site-packages/transformers/trainer.py", line 2325, in train
return inner_training_loop(
^^^^^^^^^^^^^^^^^^^^
File "/tmp/conda/wzh/lib/python3.11/site-packages/transformers/trainer.py", line 2756, in _inner_training_loop
self._maybe_log_save_evaluate(
File "/tmp/conda/wzh/lib/python3.11/site-packages/transformers/trainer.py", line 3221, in _maybe_log_save_evaluate
metrics = self._evaluate(trial, ignore_keys_for_eval)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/conda/wzh/lib/python3.11/site-packages/transformers/trainer.py", line 3170, in _evaluate
metrics = self.evaluate(ignore_keys=ignore_keys_for_eval)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/conda/wzh/lib/python3.11/site-packages/transformers/trainer.py", line 4489, in evaluate
output = eval_loop(
^^^^^^^^^^
File "/tmp/conda/wzh/lib/python3.11/site-packages/transformers/trainer.py", line 4792, in evaluation_loop
metrics[f"{metric_key_prefix}_loss"] = all_losses.mean().item()
~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'float' object does not support item assignment
[W 2025-11-26 12:22:27,448] Trial 0 failed with value None.
Traceback (most recent call last):
File "/tmp/home/wzh/final/train.py", line 160, in <module>
best_run = trainer.hyperparameter_search(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/conda/wzh/lib/python3.11/site-packages/transformers/trainer.py", line 3767, in hyperparameter_search
best_run = backend_obj.run(self, n_trials, direction, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/conda/wzh/lib/python3.11/site-packages/transformers/hyperparameter_search.py", line 72, in run
return run_hp_search_optuna(trainer, n_trials, direction, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/conda/wzh/lib/python3.11/site-packages/transformers/integrations/integration_utils.py", line 295, in run_hp_search_optuna
study.optimize(_objective, n_trials=n_trials, timeout=timeout, n_jobs=n_jobs, gc_after_trial=gc_after_trial)
File "/tmp/conda/wzh/lib/python3.11/site-packages/optuna/study/study.py", line 490, in optimize
_optimize(
File "/tmp/conda/wzh/lib/python3.11/site-packages/optuna/study/_optimize.py", line 67, in _optimize
_optimize_sequential(
File "/tmp/conda/wzh/lib/python3.11/site-packages/optuna/study/_optimize.py", line 164, in _optimize_sequential
frozen_trial_id = _run_trial(study, func, catch)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/conda/wzh/lib/python3.11/site-packages/optuna/study/_optimize.py", line 262, in _run_trial
raise func_err
File "/tmp/conda/wzh/lib/python3.11/site-packages/optuna/study/_optimize.py", line 205, in _run_trial
value_or_values = func(trial)
^^^^^^^^^^^
File "/tmp/conda/wzh/lib/python3.11/site-packages/transformers/integrations/integration_utils.py", line 277, in _objective
trainer.train(resume_from_checkpoint=checkpoint, trial=trial)
File "/tmp/conda/wzh/lib/python3.11/site-packages/transformers/trainer.py", line 2325, in train
return inner_training_loop(
^^^^^^^^^^^^^^^^^^^^
File "/tmp/conda/wzh/lib/python3.11/site-packages/transformers/trainer.py", line 2756, in _inner_training_loop
self._maybe_log_save_evaluate(
File "/tmp/conda/wzh/lib/python3.11/site-packages/transformers/trainer.py", line 3221, in _maybe_log_save_evaluate
metrics = self._evaluate(trial, ignore_keys_for_eval)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/conda/wzh/lib/python3.11/site-packages/transformers/trainer.py", line 3170, in _evaluate
metrics = self.evaluate(ignore_keys=ignore_keys_for_eval)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/conda/wzh/lib/python3.11/site-packages/transformers/trainer.py", line 4489, in evaluate
output = eval_loop(
^^^^^^^^^^
File "/tmp/conda/wzh/lib/python3.11/site-packages/transformers/trainer.py", line 4792, in evaluation_loop
metrics[f"{metric_key_prefix}_loss"] = all_losses.mean().item()
~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'float' object does not support item assignment
64%|โโโโโโโ | 1000/1565 [16:35<09:22, 1.00it/s]
[A |