File size: 72,557 Bytes
19c938f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
[NbConvertApp] Converting notebook HCP_downstream_raw_flatmaps.ipynb to python
[NbConvertApp] Writing 36673 bytes to HCP_downstream_raw_flatmaps.py
wandb: Using wandb-core as the SDK backend. Please refer to https://wandb.me/wandb-core for more information.
wandb: Currently logged in as: ckadirt. Use `wandb login --relogin` to force relogin
wandb: Tracking run with wandb version 0.18.3
wandb: Run data is saved locally in /weka/proj-fmri/ckadirt/fMRI-foundation-model/src/wandb/run-20241127_020911-HCPflat_raw_beta_trial_type_81853367-3038-4b91-805f-5066c048cef4
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run HCPflat_raw_beta_trial_type
wandb: ⭐️ View project at https://stability.wandb.io/ckadirt/fMRI-foundation-model
wandb: 🚀 View run at https://stability.wandb.io/ckadirt/fMRI-foundation-model/runs/HCPflat_raw_beta_trial_type_81853367-3038-4b91-805f-5066c048cef4
Epoch 1/50 - Training: 0%| | 0/435 [00:00<?, ?it/s]
Epoch 1/50 - Training: 0%| | 1/435 [00:26<3:13:32, 26.76s/it]
Epoch 1/50 - Training: 0%| | 2/435 [00:26<1:20:02, 11.09s/it]
Epoch 1/50 - Training: 1%| | 3/435 [00:27<43:48, 6.08s/it]
Epoch 1/50 - Training: 1%| | 4/435 [00:27<26:48, 3.73s/it]
Epoch 1/50 - Training: 1%| | 5/435 [00:27<17:25, 2.43s/it]
Epoch 1/50 - Training: 1%|▏ | 6/435 [00:27<11:50, 1.66s/it]
Epoch 1/50 - Training: 2%|▏ | 7/435 [00:27<08:16, 1.16s/it]
Epoch 1/50 - Training: 2%|▏ | 8/435 [00:27<05:57, 1.19it/s]
Epoch 1/50 - Training: 2%|▏ | 9/435 [00:27<04:23, 1.62it/s]
Epoch 1/50 - Training: 2%|▏ | 10/435 [00:27<03:20, 2.12it/s]
Epoch 1/50 - Training: 3%|▎ | 11/435 [00:44<39:04, 5.53s/it]
Epoch 1/50 - Training: 3%|▎ | 13/435 [00:45<21:00, 2.99s/it]
Epoch 1/50 - Training: 3%|▎ | 15/435 [00:45<12:48, 1.83s/it]
Epoch 1/50 - Training: 4%|▎ | 16/435 [00:45<10:07, 1.45s/it]
Epoch 1/50 - Training: 4%|▍ | 17/435 [00:45<07:53, 1.13s/it]
Epoch 1/50 - Training: 4%|▍ | 18/435 [00:45<06:07, 1.14it/s]
Epoch 1/50 - Training: 4%|▍ | 19/435 [00:45<04:43, 1.47it/s]
Epoch 1/50 - Training: 5%|▍ | 20/435 [00:45<03:41, 1.88it/s]
Epoch 1/50 - Training: 5%|▍ | 21/435 [00:55<20:17, 2.94s/it]
Epoch 1/50 - Training: 5%|▌ | 23/435 [00:55<11:23, 1.66s/it]
Epoch 1/50 - Training: 6%|▌ | 25/435 [00:55<07:11, 1.05s/it]
Epoch 1/50 - Training: 6%|▌ | 27/435 [00:55<04:52, 1.40it/s]
Epoch 1/50 - Training: 6%|▋ | 28/435 [00:55<04:03, 1.67it/s]
Epoch 1/50 - Training: 7%|▋ | 29/435 [00:55<03:19, 2.04it/s]
Epoch 1/50 - Training: 7%|▋ | 30/435 [00:56<02:45, 2.45it/s]
Epoch 1/50 - Training: 7%|▋ | 31/435 [01:01<10:54, 1.62s/it]
Epoch 1/50 - Training: 7%|▋ | 32/435 [01:01<08:08, 1.21s/it]
Epoch 1/50 - Training: 8%|▊ | 34/435 [01:01<04:51, 1.37it/s]
Epoch 1/50 - Training: 8%|▊ | 35/435 [01:01<03:52, 1.72it/s]
Epoch 1/50 - Training: 9%|▊ | 37/435 [01:01<02:33, 2.58it/s]
Epoch 1/50 - Training: 9%|▉ | 39/435 [01:01<01:50, 3.59it/s]
Epoch 1/50 - Training: 9%|▉ | 40/435 [01:02<01:36, 4.11it/s]
Epoch 1/50 - Training: 9%|▉ | 41/435 [01:07<09:20, 1.42s/it]
Epoch 1/50 - Training: 10%|▉ | 42/435 [01:07<07:13, 1.10s/it]
Epoch 1/50 - Training: 10%|▉ | 43/435 [01:07<05:30, 1.19it/s]
Epoch 1/50 - Training: 10%|█ | 45/435 [01:07<03:23, 1.92it/s]
Epoch 1/50 - Training: 11%|█ | 47/435 [01:08<02:18, 2.81it/s]
Epoch 1/50 - Training: 11%|█▏ | 49/435 [01:08<01:40, 3.84it/s]
Epoch 1/50 - Training: 12%|█▏ | 51/435 [01:13<06:57, 1.09s/it]
Epoch 1/50 - Training: 12%|█▏ | 52/435 [01:13<05:44, 1.11it/s]
Epoch 1/50 - Training: 12%|█▏ | 54/435 [01:13<03:52, 1.64it/s]
Epoch 1/50 - Training: 13%|█▎ | 56/435 [01:14<02:43, 2.32it/s]
Epoch 1/50 - Training: 13%|█▎ | 58/435 [01:14<01:59, 3.15it/s]
Epoch 1/50 - Training: 14%|█▍ | 60/435 [01:14<01:30, 4.13it/s]
Epoch 1/50 - Training: 14%|█▍ | 62/435 [01:19<06:19, 1.02s/it]
Epoch 1/50 - Training: 14%|█▍ | 63/435 [01:19<05:18, 1.17it/s]
Epoch 1/50 - Training: 15%|█▍ | 64/435 [01:20<04:20, 1.42it/s]
Epoch 1/50 - Training: 15%|█▍ | 65/435 [01:20<03:30, 1.75it/s]
Epoch 1/50 - Training: 15%|█▌ | 66/435 [01:20<02:49, 2.18it/s]
Epoch 1/50 - Training: 15%|█▌ | 67/435 [01:20<02:15, 2.71it/s]
Epoch 1/50 - Training: 16%|█▌ | 68/435 [01:20<01:49, 3.34it/s]
Epoch 1/50 - Training: 16%|█▌ | 69/435 [01:20<01:30, 4.05it/s]
Epoch 1/50 - Training: 16%|█▌ | 70/435 [01:20<01:15, 4.81it/s]
Epoch 1/50 - Training: 16%|█▋ | 71/435 [01:26<11:29, 1.89s/it]
Epoch 1/50 - Training: 17%|█▋ | 72/435 [01:26<08:18, 1.37s/it]
Epoch 1/50 - Training: 17%|█▋ | 73/435 [01:27<06:03, 1.00s/it]
Epoch 1/50 - Training: 17%|█▋ | 74/435 [01:27<04:27, 1.35it/s]
Epoch 1/50 - Training: 17%|█▋ | 75/435 [01:27<03:20, 1.79it/s]
Epoch 1/50 - Training: 17%|█▋ | 76/435 [01:27<02:33, 2.34it/s]
Epoch 1/50 - Training: 18%|█▊ | 77/435 [01:27<02:00, 2.98it/s]
Epoch 1/50 - Training: 18%|█▊ | 78/435 [01:27<01:35, 3.74it/s]
Epoch 1/50 - Training: 18%|█▊ | 79/435 [01:27<01:19, 4.49it/s]
Epoch 1/50 - Training: 18%|█▊ | 80/435 [01:27<01:08, 5.18it/s]
Epoch 1/50 - Training: 19%|█▊ | 81/435 [01:33<11:16, 1.91s/it]
Epoch 1/50 - Training: 19%|█▉ | 82/435 [01:33<08:04, 1.37s/it]
Epoch 1/50 - Training: 19%|█▉ | 83/435 [01:34<05:49, 1.01it/s]
Epoch 1/50 - Training: 19%|█▉ | 84/435 [01:34<04:17, 1.37it/s]
Epoch 1/50 - Training: 20%|█▉ | 85/435 [01:34<03:13, 1.81it/s]
Epoch 1/50 - Training: 20%|█▉ | 86/435 [01:34<02:27, 2.36it/s]
Epoch 1/50 - Training: 20%|██ | 87/435 [01:34<01:55, 3.02it/s]
Epoch 1/50 - Training: 20%|██ | 89/435 [01:34<01:17, 4.46it/s]
Epoch 1/50 - Training: 21%|██ | 90/435 [01:34<01:08, 5.06it/s]
Epoch 1/50 - Training: 21%|██ | 91/435 [01:41<10:09, 1.77s/it]
Epoch 1/50 - Training: 21%|██ | 92/435 [01:41<07:32, 1.32s/it]
Epoch 1/50 - Training: 21%|██▏ | 93/435 [01:41<05:35, 1.02it/s]
Epoch 1/50 - Training: 22%|██▏ | 94/435 [01:41<04:08, 1.37it/s]
Epoch 1/50 - Training: 22%|██▏ | 95/435 [01:41<03:06, 1.82it/s]
Epoch 1/50 - Training: 22%|██▏ | 96/435 [01:41<02:22, 2.37it/s]
Epoch 1/50 - Training: 22%|██▏ | 97/435 [01:41<01:51, 3.04it/s]
Epoch 1/50 - Training: 23%|██▎ | 99/435 [01:41<01:14, 4.49it/s]
Epoch 1/50 - Training: 23%|██▎ | 100/435 [01:42<01:07, 5.00it/s]
Epoch 1/50 - Training: 23%|██▎ | 101/435 [01:48<10:03, 1.81s/it]
Epoch 1/50 - Training: 23%|██▎ | 102/435 [01:48<07:29, 1.35s/it]
Epoch 1/50 - Training: 24%|██▎ | 103/435 [01:48<05:32, 1.00s/it]
Epoch 1/50 - Training: 24%|██▍ | 104/435 [01:48<04:06, 1.34it/s]
Epoch 1/50 - Training: 24%|██▍ | 105/435 [01:49<03:56, 1.40it/s]
Epoch 1/50 - Training: 24%|██▍ | 106/435 [01:49<02:59, 1.83it/s]
Epoch 1/50 - Training: 25%|██▍ | 107/435 [01:49<02:18, 2.36it/s]
Epoch 1/50 - Training: 25%|██▍ | 108/435 [01:49<01:47, 3.03it/s]
Epoch 1/50 - Training: 25%|██▌ | 109/435 [01:49<01:26, 3.78it/s]
Epoch 1/50 - Training: 25%|██▌ | 110/435 [01:49<01:13, 4.43it/s]
Epoch 1/50 - Training: 26%|██▌ | 111/435 [01:55<10:07, 1.88s/it]
Epoch 1/50 - Training: 26%|██▌ | 112/435 [01:55<07:14, 1.35s/it]
Epoch 1/50 - Training: 26%|██▌ | 113/435 [01:55<05:13, 1.03it/s]
Epoch 1/50 - Training: 26%|██▌ | 114/435 [01:56<04:47, 1.12it/s]
Epoch 1/50 - Training: 26%|██▋ | 115/435 [01:57<05:00, 1.06it/s]
Epoch 1/50 - Training: 27%|██▋ | 116/435 [01:57<03:40, 1.45it/s]
Epoch 1/50 - Training: 27%|██▋ | 117/435 [01:57<02:47, 1.89it/s]
Epoch 1/50 - Training: 27%|██▋ | 118/435 [01:58<02:07, 2.48it/s]
Epoch 1/50 - Training: 27%|██▋ | 119/435 [01:58<01:42, 3.10it/s]
Epoch 1/50 - Training: 28%|██▊ | 120/435 [01:58<01:24, 3.74it/s]
Epoch 1/50 - Training: 28%|██▊ | 121/435 [02:03<08:30, 1.63s/it]
Epoch 1/50 - Training: 28%|██▊ | 122/435 [02:03<06:06, 1.17s/it]
Epoch 1/50 - Training: 28%|██▊ | 123/435 [02:03<04:30, 1.16it/s]
Epoch 1/50 - Training: 29%|██▊ | 124/435 [02:05<05:46, 1.11s/it]
Epoch 1/50 - Training: 29%|██▊ | 125/435 [02:06<05:40, 1.10s/it]
Epoch 1/50 - Training: 29%|██▉ | 126/435 [02:06<04:08, 1.24it/s]
Epoch 1/50 - Training: 29%|██▉ | 127/435 [02:06<03:15, 1.58it/s]
Epoch 1/50 - Training: 29%|██▉ | 128/435 [02:06<02:26, 2.10it/s]
Epoch 1/50 - Training: 30%|██▉ | 129/435 [02:06<01:52, 2.73it/s]
Epoch 1/50 - Training: 30%|██▉ | 130/435 [02:06<01:29, 3.41it/s]
Epoch 1/50 - Training: 30%|███ | 131/435 [02:10<06:38, 1.31s/it]
Epoch 1/50 - Training: 30%|███ | 132/435 [02:10<04:49, 1.05it/s]
Epoch 1/50 - Training: 31%|███ | 133/435 [02:11<04:56, 1.02it/s]
Epoch 1/50 - Training: 31%|███ | 134/435 [02:13<06:13, 1.24s/it]
Epoch 1/50 - Training: 31%|███ | 135/435 [02:14<05:49, 1.16s/it]
Epoch 1/50 - Training: 31%|███▏ | 136/435 [02:14<04:15, 1.17it/s]
Epoch 1/50 - Training: 31%|███▏ | 137/435 [02:15<03:38, 1.37it/s]
Epoch 1/50 - Training: 32%|███▏ | 138/435 [02:15<02:42, 1.83it/s]
Epoch 1/50 - Training: 32%|███▏ | 139/435 [02:15<02:03, 2.41it/s]
Epoch 1/50 - Training: 32%|███▏ | 140/435 [02:15<01:37, 3.02it/s]
Epoch 1/50 - Training: 32%|███▏ | 141/435 [02:17<04:45, 1.03it/s]
Epoch 1/50 - Training: 33%|███▎ | 142/435 [02:18<04:11, 1.17it/s]
Epoch 1/50 - Training: 33%|███▎ | 143/435 [02:19<04:58, 1.02s/it]
Epoch 1/50 - Training: 33%|███▎ | 144/435 [02:21<06:25, 1.33s/it]
Epoch 1/50 - Training: 33%|███▎ | 145/435 [02:22<05:49, 1.20s/it]
Epoch 1/50 - Training: 34%|███▎ | 146/435 [02:23<04:16, 1.13it/s]
Epoch 1/50 - Training: 34%|███▍ | 147/435 [02:23<03:51, 1.24it/s]
Epoch 1/50 - Training: 34%|███▍ | 148/435 [02:23<02:51, 1.68it/s]
Epoch 1/50 - Training: 34%|███▍ | 149/435 [02:23<02:09, 2.21it/s]
Epoch 1/50 - Training: 34%|███▍ | 150/435 [02:24<01:40, 2.82it/s]
Epoch 1/50 - Training: 35%|███▍ | 151/435 [02:25<03:01, 1.57it/s]
Epoch 1/50 - Training: 35%|███▍ | 152/435 [02:26<04:04, 1.16it/s]
Epoch 1/50 - Training: 35%|███▌ | 153/435 [02:28<04:57, 1.05s/it]
Epoch 1/50 - Training: 35%|███▌ | 154/435 [02:30<06:35, 1.41s/it]
Epoch 1/50 - Training: 36%|███▌ | 155/435 [02:31<05:48, 1.24s/it]
Epoch 1/50 - Training: 36%|███▌ | 156/435 [02:31<04:13, 1.10it/s]
Epoch 1/50 - Training: 36%|███▌ | 157/435 [02:32<04:03, 1.14it/s]
Epoch 1/50 - Training: 37%|███▋ | 159/435 [02:32<02:22, 1.93it/s]
Epoch 1/50 - Training: 37%|███▋ | 160/435 [02:32<01:55, 2.38it/s]
Epoch 1/50 - Training: 37%|███▋ | 161/435 [02:32<01:37, 2.82it/s]
Epoch 1/50 - Training: 37%|███▋ | 162/435 [02:34<03:50, 1.19it/s]
Epoch 1/50 - Training: 37%|███▋ | 163/435 [02:36<04:46, 1.05s/it]
Epoch 1/50 - Training: 38%|███▊ | 164/435 [02:38<06:32, 1.45s/it]
Epoch 1/50 - Training: 38%|███▊ | 165/435 [02:39<05:40, 1.26s/it]
Epoch 1/50 - Training: 38%|███▊ | 166/435 [02:39<04:10, 1.07it/s]
Epoch 1/50 - Training: 38%|███▊ | 167/435 [02:40<03:50, 1.16it/s]
Epoch 1/50 - Training: 39%|███▊ | 168/435 [02:40<02:50, 1.57it/s]
Epoch 1/50 - Training: 39%|███▉ | 169/435 [02:40<02:07, 2.09it/s]
Epoch 1/50 - Training: 39%|███▉ | 170/435 [02:40<01:38, 2.69it/s]
Epoch 1/50 - Training: 40%|███▉ | 172/435 [02:42<02:52, 1.52it/s]
Epoch 1/50 - Training: 40%|███▉ | 173/435 [02:44<03:58, 1.10it/s]
Epoch 1/50 - Training: 40%|████ | 174/435 [02:47<05:49, 1.34s/it]
Epoch 1/50 - Training: 40%|████ | 175/435 [02:47<05:05, 1.18s/it]
Epoch 1/50 - Training: 40%|████ | 176/435 [02:47<03:49, 1.13it/s]
Epoch 1/50 - Training: 41%|████ | 177/435 [02:48<03:59, 1.08it/s]
Epoch 1/50 - Training: 41%|████ | 179/435 [02:49<02:21, 1.80it/s]
Epoch 1/50 - Training: 41%|████▏ | 180/435 [02:49<01:54, 2.23it/s]
Epoch 1/50 - Training: 42%|████▏ | 182/435 [02:50<02:26, 1.73it/s]
Epoch 1/50 - Training: 42%|████▏ | 183/435 [02:52<03:38, 1.16it/s]
Epoch 1/50 - Training: 42%|████▏ | 184/435 [02:55<05:21, 1.28s/it]
Epoch 1/50 - Training: 43%|████▎ | 185/435 [02:55<04:45, 1.14s/it]
Epoch 1/50 - Training: 43%|████▎ | 186/435 [02:56<03:36, 1.15it/s]
Epoch 1/50 - Training: 43%|████▎ | 187/435 [02:57<04:00, 1.03it/s]
Epoch 1/50 - Training: 43%|████▎ | 189/435 [02:57<02:22, 1.72it/s]
Epoch 1/50 - Training: 44%|████▎ | 190/435 [02:57<01:54, 2.15it/s]
Epoch 1/50 - Training: 44%|████▍ | 192/435 [02:58<02:06, 1.92it/s]
Epoch 1/50 - Training: 44%|████▍ | 193/435 [03:00<03:22, 1.19it/s]
Epoch 1/50 - Training: 45%|████▍ | 194/435 [03:03<05:02, 1.26s/it]
Epoch 1/50 - Training: 45%|████▍ | 195/435 [03:03<04:21, 1.09s/it]
Epoch 1/50 - Training: 45%|████▌ | 196/435 [03:04<03:18, 1.21it/s]
Epoch 1/50 - Training: 45%|████▌ | 197/435 [03:05<04:01, 1.01s/it]
Epoch 1/50 - Training: 46%|████▌ | 199/435 [03:05<02:23, 1.65it/s]
Epoch 1/50 - Training: 46%|████▌ | 200/435 [03:05<01:54, 2.06it/s]
Epoch 1/50 - Training: 46%|████▋ | 202/435 [03:06<01:56, 1.99it/s]
Epoch 1/50 - Training: 47%|████▋ | 203/435 [03:08<02:31, 1.53it/s]
Epoch 1/50 - Training: 47%|████▋ | 204/435 [03:11<05:08, 1.34s/it]
Epoch 1/50 - Training: 47%|████▋ | 205/435 [03:12<04:49, 1.26s/it]
Epoch 1/50 - Training: 48%|████▊ | 207/435 [03:14<03:57, 1.04s/it]
Epoch 1/50 - Training: 48%|████▊ | 209/435 [03:14<02:34, 1.46it/s]
Epoch 1/50 - Training: 48%|████▊ | 210/435 [03:14<02:06, 1.78it/s]
Epoch 1/50 - Training: 49%|████▊ | 212/435 [03:15<02:06, 1.76it/s]
Epoch 1/50 - Training: 49%|████▉ | 213/435 [03:16<02:16, 1.62it/s]
Epoch 1/50 - Training: 49%|████▉ | 214/435 [03:18<04:00, 1.09s/it]
Epoch 1/50 - Training: 49%|████▉ | 215/435 [03:20<04:38, 1.27s/it]
Epoch 1/50 - Training: 50%|████▉ | 216/435 [03:20<03:31, 1.04it/s]
Epoch 1/50 - Training: 50%|████▉ | 217/435 [03:21<03:08, 1.16it/s]
Epoch 1/50 - Training: 50%|█████ | 219/435 [03:22<02:27, 1.46it/s]
Epoch 1/50 - Training: 51%|█████ | 220/435 [03:22<01:59, 1.81it/s]
Epoch 1/50 - Training: 51%|█████ | 222/435 [03:23<01:56, 1.83it/s]
Epoch 1/50 - Training: 51%|█████▏ | 223/435 [03:24<02:01, 1.75it/s]
Epoch 1/50 - Training: 51%|█████▏ | 224/435 [03:26<03:40, 1.04s/it]
Epoch 1/50 - Training: 52%|█████▏ | 225/435 [03:30<05:42, 1.63s/it]
Epoch 1/50 - Training: 52%|█████▏ | 226/435 [03:30<04:16, 1.23s/it]
Epoch 1/50 - Training: 52%|█████▏ | 227/435 [03:30<03:11, 1.09it/s]
Epoch 1/50 - Training: 53%|█████▎ | 229/435 [03:32<03:04, 1.12it/s]
Epoch 1/50 - Training: 53%|█████▎ | 230/435 [03:32<02:25, 1.41it/s]
Epoch 1/50 - Training: 53%|█████▎ | 231/435 [03:32<01:53, 1.79it/s]
Epoch 1/50 - Training: 53%|█████▎ | 232/435 [03:33<02:08, 1.58it/s]
Epoch 1/50 - Training: 54%|█████▎ | 233/435 [03:33<02:03, 1.63it/s]
Epoch 1/50 - Training: 54%|█████▍ | 234/435 [03:34<02:38, 1.27it/s]
Epoch 1/50 - Training: 54%|█████▍ | 235/435 [03:39<06:23, 1.92s/it]
Epoch 1/50 - Training: 54%|█████▍ | 237/435 [03:39<03:37, 1.10s/it]
Epoch 1/50 - Training: 55%|█████▍ | 239/435 [03:41<03:05, 1.05it/s]
Epoch 1/50 - Training: 55%|█████▌ | 241/435 [03:41<02:03, 1.57it/s]
Epoch 1/50 - Training: 56%|█████▌ | 242/435 [03:42<02:21, 1.36it/s]
Epoch 1/50 - Training: 56%|█████▌ | 244/435 [03:43<02:03, 1.55it/s]
Epoch 1/50 - Training: 56%|█████▋ | 245/435 [03:48<04:53, 1.54s/it]
Epoch 1/50 - Training: 57%|█████▋ | 247/435 [03:48<03:08, 1.00s/it]
Epoch 1/50 - Training: 57%|█████▋ | 249/435 [03:49<02:26, 1.27it/s]
Epoch 1/50 - Training: 58%|█████▊ | 251/435 [03:49<01:42, 1.79it/s]
Epoch 1/50 - Training: 58%|█████▊ | 252/435 [04:00<07:33, 2.48s/it]
Epoch 1/50 - Training: 58%|█████▊ | 254/435 [04:00<04:55, 1.63s/it]
Epoch 1/50 - Training: 59%|█████▉ | 256/435 [04:00<03:20, 1.12s/it]
Epoch 1/50 - Training: 59%|█████▉ | 258/435 [04:00<02:19, 1.27it/s]
Epoch 1/50 - Training: 60%|█████▉ | 260/435 [04:00<01:39, 1.77it/s]
Epoch 1/50 - Training: 60%|██████ | 261/435 [04:01<01:23, 2.08it/s]
Epoch 1/50 - Training: 60%|██████ | 262/435 [04:07<04:40, 1.62s/it]
Epoch 1/50 - Training: 61%|██████ | 264/435 [04:07<02:59, 1.05s/it]
Epoch 1/50 - Training: 61%|██████ | 266/435 [04:07<02:01, 1.39it/s]
Epoch 1/50 - Training: 62%|██████▏ | 268/435 [04:07<01:25, 1.96it/s]
Epoch 1/50 - Training: 62%|██████▏ | 269/435 [04:07<01:11, 2.31it/s]
Epoch 1/50 - Training: 62%|██████▏ | 271/435 [04:07<00:51, 3.18it/s]
Epoch 1/50 - Training: 63%|██████▎ | 272/435 [04:16<05:09, 1.90s/it]
Epoch 1/50 - Training: 63%|██████▎ | 274/435 [04:16<03:19, 1.24s/it]
Epoch 1/50 - Training: 63%|██████▎ | 275/435 [04:16<02:40, 1.01s/it]
Epoch 1/50 - Training: 64%|██████▎ | 277/435 [04:16<01:45, 1.50it/s]
Epoch 1/50 - Training: 64%|██████▍ | 278/435 [04:16<01:26, 1.82it/s]
Epoch 1/50 - Training: 64%|██████▍ | 279/435 [04:16<01:09, 2.24it/s]
Epoch 1/50 - Training: 65%|██████▍ | 281/435 [04:16<00:47, 3.24it/s]
Epoch 1/50 - Training: 65%|██████▍ | 282/435 [04:24<05:04, 1.99s/it]
Epoch 1/50 - Training: 65%|██████▌ | 284/435 [04:25<03:09, 1.26s/it]
Epoch 1/50 - Training: 66%|██████▌ | 286/435 [04:25<02:05, 1.18it/s]
Epoch 1/50 - Training: 66%|██████▌ | 287/435 [04:25<01:43, 1.43it/s]
Epoch 1/50 - Training: 66%|██████▌ | 288/435 [04:25<01:24, 1.75it/s]
Epoch 1/50 - Training: 66%|██████▋ | 289/435 [04:25<01:07, 2.17it/s]
Epoch 1/50 - Training: 67%|██████▋ | 291/435 [04:25<00:45, 3.16it/s]
Epoch 1/50 - Training: 67%|██████▋ | 292/435 [04:33<04:38, 1.95s/it]
Epoch 1/50 - Training: 68%|██████▊ | 294/435 [04:33<02:52, 1.22s/it]
Epoch 1/50 - Training: 68%|██████▊ | 296/435 [04:33<01:53, 1.23it/s]
Epoch 1/50 - Training: 69%|██████▊ | 298/435 [04:33<01:18, 1.75it/s]
Epoch 1/50 - Training: 69%|██████▉ | 300/435 [04:34<00:56, 2.39it/s]
Epoch 1/50 - Training: 69%|██████▉ | 301/435 [04:34<00:48, 2.78it/s]
Epoch 1/50 - Training: 69%|██████▉ | 302/435 [04:42<04:24, 1.99s/it]
Epoch 1/50 - Training: 70%|██████▉ | 304/435 [04:42<02:46, 1.27s/it]
Epoch 1/50 - Training: 70%|███████ | 305/435 [04:42<02:13, 1.03s/it]
Epoch 1/50 - Training: 71%|███████ | 307/435 [04:42<01:26, 1.49it/s]
Epoch 1/50 - Training: 71%|███████ | 308/435 [04:42<01:09, 1.81it/s]
Epoch 1/50 - Training: 71%|███████ | 309/435 [04:42<00:56, 2.24it/s]
Epoch 1/50 - Training: 71%|███████▏ | 311/435 [04:43<00:37, 3.27it/s]
Epoch 1/50 - Training: 72%|███████▏ | 312/435 [04:51<04:13, 2.06s/it]
Epoch 1/50 - Training: 72%|███████▏ | 314/435 [04:51<02:37, 1.30s/it]
Epoch 1/50 - Training: 72%|███████▏ | 315/435 [04:51<02:05, 1.05s/it]
Epoch 1/50 - Training: 73%|███████▎ | 317/435 [04:51<01:20, 1.46it/s]
Epoch 1/50 - Training: 73%|███████▎ | 318/435 [04:52<01:06, 1.76it/s]
Epoch 1/50 - Training: 73%|███████▎ | 319/435 [04:52<00:53, 2.17it/s]
Epoch 1/50 - Training: 74%|███████▎ | 320/435 [04:52<00:42, 2.69it/s]
Epoch 1/50 - Training: 74%|███████▍ | 321/435 [04:52<00:34, 3.29it/s]
Epoch 1/50 - Training: 74%|███████▍ | 322/435 [04:59<03:59, 2.12s/it]
Epoch 1/50 - Training: 74%|███████▍ | 324/435 [04:59<02:15, 1.22s/it]
Epoch 1/50 - Training: 75%|███████▍ | 325/435 [04:59<01:45, 1.04it/s]
Epoch 1/50 - Training: 75%|███████▌ | 327/435 [04:59<01:05, 1.65it/s]
Epoch 1/50 - Training: 76%|███████▌ | 329/435 [04:59<00:44, 2.39it/s]
Epoch 1/50 - Training: 76%|███████▌ | 330/435 [05:00<00:37, 2.82it/s]
Epoch 1/50 - Training: 76%|███████▋ | 332/435 [05:07<02:43, 1.59s/it]
Epoch 1/50 - Training: 77%|███████▋ | 334/435 [05:07<01:48, 1.07s/it]
Epoch 1/50 - Training: 77%|███████▋ | 335/435 [05:07<01:28, 1.13it/s]
Epoch 1/50 - Training: 77%|███████▋ | 336/435 [05:07<01:10, 1.40it/s]
Epoch 1/50 - Training: 77%|███████▋ | 337/435 [05:07<00:55, 1.76it/s]
Epoch 1/50 - Training: 78%|███████▊ | 338/435 [05:08<00:43, 2.23it/s]
Epoch 1/50 - Training: 78%|███████▊ | 339/435 [05:08<00:34, 2.82it/s]
Epoch 1/50 - Training: 78%|███████▊ | 341/435 [05:08<00:23, 4.08it/s]
Epoch 1/50 - Training: 79%|███████▊ | 342/435 [05:16<03:14, 2.09s/it]
Epoch 1/50 - Training: 79%|███████▉ | 344/435 [05:16<01:56, 1.28s/it]
Epoch 1/50 - Training: 79%|███████▉ | 345/435 [05:17<01:46, 1.19s/it]
Epoch 1/50 - Training: 80%|███████▉ | 347/435 [05:17<01:07, 1.30it/s]
Epoch 1/50 - Training: 80%|████████ | 348/435 [05:17<00:54, 1.60it/s]
Epoch 1/50 - Training: 80%|████████ | 350/435 [05:17<00:35, 2.37it/s]
Epoch 1/50 - Training: 81%|████████ | 352/435 [05:24<01:59, 1.44s/it]
Epoch 1/50 - Training: 81%|████████▏ | 354/435 [05:24<01:19, 1.02it/s]
Epoch 1/50 - Training: 82%|████████▏ | 355/435 [05:26<01:37, 1.22s/it]
Epoch 1/50 - Training: 82%|████████▏ | 357/435 [05:27<01:04, 1.22it/s]
Epoch 1/50 - Training: 83%|████████▎ | 359/435 [05:27<00:43, 1.73it/s]
Epoch 1/50 - Training: 83%|████████▎ | 360/435 [05:27<00:36, 2.06it/s]
Epoch 1/50 - Training: 83%|████████▎ | 361/435 [05:27<00:29, 2.48it/s]
Epoch 1/50 - Training: 83%|████████▎ | 362/435 [05:32<01:54, 1.57s/it]
Epoch 1/50 - Training: 83%|████████▎ | 363/435 [05:32<01:26, 1.20s/it]
Epoch 1/50 - Training: 84%|████████▎ | 364/435 [05:33<01:04, 1.10it/s]
Epoch 1/50 - Training: 84%|████████▍ | 365/435 [05:36<01:50, 1.58s/it]
Epoch 1/50 - Training: 84%|████████▍ | 366/435 [05:36<01:20, 1.16s/it]
Epoch 1/50 - Training: 85%|████████▍ | 368/435 [05:36<00:46, 1.45it/s]
Epoch 1/50 - Training: 85%|████████▍ | 369/435 [05:36<00:36, 1.82it/s]
Epoch 1/50 - Training: 85%|████████▌ | 370/435 [05:36<00:28, 2.28it/s]
Epoch 1/50 - Training: 86%|████████▌ | 372/435 [05:40<01:05, 1.04s/it]
Epoch 1/50 - Training: 86%|████████▌ | 374/435 [05:40<00:41, 1.46it/s]
Epoch 1/50 - Training: 86%|████████▌ | 375/435 [05:45<01:28, 1.48s/it]
Epoch 1/50 - Training: 87%|████████▋ | 377/435 [05:45<00:55, 1.05it/s]
Epoch 1/50 - Training: 87%|████████▋ | 379/435 [05:45<00:36, 1.53it/s]
Epoch 1/50 - Training: 87%|████████▋ | 380/435 [05:45<00:30, 1.83it/s]
Epoch 1/50 - Training: 88%|████████▊ | 382/435 [05:48<00:44, 1.19it/s]
Epoch 1/50 - Training: 88%|████████▊ | 383/435 [05:48<00:35, 1.45it/s]
Epoch 1/50 - Training: 88%|████████▊ | 384/435 [05:48<00:28, 1.79it/s]
Epoch 1/50 - Training: 89%|████████▊ | 385/435 [05:53<01:22, 1.64s/it]
Epoch 1/50 - Training: 89%|████████▉ | 387/435 [05:53<00:48, 1.01s/it]
Epoch 1/50 - Training: 89%|████████▉ | 388/435 [05:53<00:37, 1.25it/s]
Epoch 1/50 - Training: 89%|████████▉ | 389/435 [05:54<00:29, 1.58it/s]
Epoch 1/50 - Training: 90%|████████▉ | 390/435 [05:54<00:22, 2.01it/s]
Epoch 1/50 - Training: 90%|████████▉ | 391/435 [05:54<00:17, 2.55it/s]
Epoch 1/50 - Training: 90%|█████████ | 392/435 [05:55<00:26, 1.64it/s]
Epoch 1/50 - Training: 90%|█████████ | 393/435 [05:55<00:19, 2.14it/s]
Epoch 1/50 - Training: 91%|█████████ | 394/435 [05:55<00:15, 2.71it/s]
Epoch 1/50 - Training: 91%|█████████ | 395/435 [06:02<01:31, 2.28s/it]
Epoch 1/50 - Training: 91%|█████████▏| 397/435 [06:02<00:48, 1.28s/it]
Epoch 1/50 - Training: 91%|█████████▏| 398/435 [06:02<00:37, 1.00s/it]
Epoch 1/50 - Training: 92%|█████████▏| 399/435 [06:03<00:27, 1.30it/s]
Epoch 1/50 - Training: 92%|█████████▏| 400/435 [06:03<00:20, 1.68it/s]
Epoch 1/50 - Training: 92%|█████████▏| 401/435 [06:03<00:15, 2.17it/s]
Epoch 1/50 - Training: 92%|█████████▏| 402/435 [06:03<00:11, 2.77it/s]
Epoch 1/50 - Training: 93%|█████████▎| 403/435 [06:03<00:09, 3.42it/s]
Epoch 1/50 - Training: 93%|█████████▎| 404/435 [06:03<00:07, 4.10it/s]
Epoch 1/50 - Training: 93%|█████████▎| 405/435 [06:12<01:24, 2.81s/it]
Epoch 1/50 - Training: 94%|█████████▎| 407/435 [06:12<00:43, 1.57s/it]
Epoch 1/50 - Training: 94%|█████████▍| 408/435 [06:12<00:32, 1.21s/it]
Epoch 1/50 - Training: 94%|█████████▍| 409/435 [06:13<00:24, 1.07it/s]
Epoch 1/50 - Training: 94%|█████████▍| 410/435 [06:13<00:17, 1.39it/s]
Epoch 1/50 - Training: 94%|█████████▍| 411/435 [06:13<00:13, 1.82it/s]
Epoch 1/50 - Training: 95%|█████████▍| 412/435 [06:13<00:09, 2.35it/s]
Epoch 1/50 - Training: 95%|█████████▍| 413/435 [06:13<00:07, 2.99it/s]
Epoch 1/50 - Training: 95%|█████████▌| 414/435 [06:13<00:05, 3.70it/s]
Epoch 1/50 - Training: 95%|█████████▌| 415/435 [06:21<00:52, 2.61s/it]
Epoch 1/50 - Training: 96%|█████████▌| 416/435 [06:21<00:35, 1.87s/it]
Epoch 1/50 - Training: 96%|█████████▌| 418/435 [06:22<00:17, 1.05s/it]
Epoch 1/50 - Training: 97%|█████████▋| 420/435 [06:22<00:10, 1.49it/s]
Epoch 1/50 - Training: 97%|█████████▋| 422/435 [06:22<00:05, 2.19it/s]
Epoch 1/50 - Training: 97%|█████████▋| 424/435 [06:22<00:03, 3.00it/s]
Epoch 1/50 - Training: 98%|█████████▊| 426/435 [06:26<00:08, 1.08it/s]
Epoch 1/50 - Training: 98%|█████████▊| 428/435 [06:27<00:04, 1.52it/s]
Epoch 1/50 - Training: 99%|█████████▉| 430/435 [06:27<00:02, 2.09it/s]
Epoch 1/50 - Training: 99%|█████████▉| 432/435 [06:27<00:01, 2.79it/s]
Epoch 1/50 - Training: 100%|█████████▉| 434/435 [06:27<00:00, 3.61it/s]
Epoch 1/50 - Training: 100%|██████████| 435/435 [06:30<00:00, 1.11it/s]
Epoch 1/50 - Validation: 0%| | 0/48 [00:00<?, ?it/s]
Epoch 1/50 - Validation: 2%|▏ | 1/48 [00:03<03:00, 3.83s/it]
Epoch 1/50 - Validation: 4%|▍ | 2/48 [00:07<02:43, 3.56s/it]
Epoch 1/50 - Validation: 6%|▋ | 3/48 [00:10<02:41, 3.60s/it]
Epoch 1/50 - Validation: 8%|▊ | 4/48 [00:14<02:34, 3.52s/it]
Epoch 1/50 - Validation: 10%|█ | 5/48 [00:17<02:27, 3.44s/it]
Epoch 1/50 - Validation: 12%|█▎ | 6/48 [00:20<02:23, 3.41s/it]
Epoch 1/50 - Validation: 15%|█▍ | 7/48 [00:24<02:17, 3.34s/it]
Epoch 1/50 - Validation: 17%|█▋ | 8/48 [00:27<02:15, 3.38s/it]
Epoch 1/50 - Validation: 19%|█▉ | 9/48 [00:30<02:06, 3.25s/it]
Epoch 1/50 - Validation: 21%|██ | 10/48 [00:33<02:02, 3.24s/it]
Epoch 1/50 - Validation: 23%|██▎ | 11/48 [00:37<02:01, 3.27s/it]
Epoch 1/50 - Validation: 25%|██▌ | 12/48 [00:40<01:54, 3.17s/it]
Epoch 1/50 - Validation: 27%|██▋ | 13/48 [00:43<01:49, 3.13s/it]
Epoch 1/50 - Validation: 29%|██▉ | 14/48 [00:45<01:44, 3.07s/it]
Epoch 1/50 - Validation: 31%|███▏ | 15/48 [00:48<01:38, 2.99s/it]
Epoch 1/50 - Validation: 33%|███▎ | 16/48 [00:51<01:35, 2.98s/it]
Epoch 1/50 - Validation: 35%|███▌ | 17/48 [00:54<01:30, 2.91s/it]
Epoch 1/50 - Validation: 38%|███▊ | 18/48 [00:57<01:25, 2.86s/it]
Epoch 1/50 - Validation: 40%|███▉ | 19/48 [01:00<01:26, 2.98s/it]
Epoch 1/50 - Validation: 42%|████▏ | 20/48 [01:03<01:23, 2.99s/it]
Epoch 1/50 - Validation: 44%|████▍ | 21/48 [01:06<01:19, 2.96s/it]
Epoch 1/50 - Validation: 46%|████▌ | 22/48 [01:09<01:14, 2.87s/it]
Epoch 1/50 - Validation: 48%|████▊ | 23/48 [01:11<01:09, 2.76s/it]
Epoch 1/50 - Validation: 50%|█████ | 24/48 [01:14<01:07, 2.80s/it]
Epoch 1/50 - Validation: 52%|█████▏ | 25/48 [01:17<01:03, 2.74s/it]
Epoch 1/50 - Validation: 54%|█████▍ | 26/48 [01:19<00:59, 2.70s/it]
Epoch 1/50 - Validation: 56%|█████▋ | 27/48 [01:22<00:55, 2.66s/it]
Epoch 1/50 - Validation: 58%|█████▊ | 28/48 [01:24<00:52, 2.61s/it]
Epoch 1/50 - Validation: 60%|██████ | 29/48 [01:27<00:49, 2.59s/it]
Epoch 1/50 - Validation: 62%|██████▎ | 30/48 [01:29<00:45, 2.54s/it]
Epoch 1/50 - Validation: 65%|██████▍ | 31/48 [01:32<00:43, 2.58s/it]
Epoch 1/50 - Validation: 67%|██████▋ | 32/48 [01:34<00:41, 2.57s/it]
Epoch 1/50 - Validation: 69%|██████▉ | 33/48 [01:37<00:38, 2.54s/it]
Epoch 1/50 - Validation: 71%|███████ | 34/48 [01:39<00:35, 2.53s/it]
Epoch 1/50 - Validation: 73%|███████▎ | 35/48 [01:42<00:33, 2.57s/it]
Epoch 1/50 - Validation: 75%|███████▌ | 36/48 [01:45<00:30, 2.54s/it]
Epoch 1/50 - Validation: 77%|███████▋ | 37/48 [01:47<00:27, 2.51s/it]
Epoch 1/50 - Validation: 79%|███████▉ | 38/48 [01:50<00:26, 2.66s/it]
Epoch 1/50 - Validation: 81%|████████▏ | 39/48 [01:52<00:23, 2.61s/it]
Epoch 1/50 - Validation: 83%|████████▎ | 40/48 [01:55<00:20, 2.56s/it]
Epoch 1/50 - Validation: 85%|████████▌ | 41/48 [01:57<00:17, 2.57s/it]
Epoch 1/50 - Validation: 88%|████████▊ | 42/48 [02:00<00:15, 2.54s/it]
Epoch 1/50 - Validation: 90%|████████▉ | 43/48 [02:02<00:12, 2.51s/it]
Epoch 1/50 - Validation: 92%|█████████▏| 44/48 [02:05<00:10, 2.52s/it]
Epoch 1/50 - Validation: 94%|█████████▍| 45/48 [02:07<00:07, 2.51s/it]
Epoch 1/50 - Validation: 96%|█████████▌| 46/48 [02:10<00:04, 2.50s/it]
Epoch 1/50 - Validation: 98%|█████████▊| 47/48 [02:12<00:02, 2.46s/it]
Epoch 1/50 - Validation: 100%|██████████| 48/48 [02:13<00:00, 1.87s/it]
Epoch 1/50 - Validation: 100%|██████████| 48/48 [02:13<00:00, 2.78s/it]
Epoch 2/50 - Training: 0%| | 0/435 [00:00<?, ?it/s]
Epoch 2/50 - Training: 0%| | 1/435 [00:12<1:29:40, 12.40s/it]
Epoch 2/50 - Training: 0%| | 2/435 [00:12<37:17, 5.17s/it]
Epoch 2/50 - Training: 1%| | 3/435 [00:12<20:35, 2.86s/it]
Epoch 2/50 - Training: 1%| | 4/435 [00:12<12:43, 1.77s/it]
Epoch 2/50 - Training: 1%| | 5/435 [00:12<08:23, 1.17s/it]
Epoch 2/50 - Training: 1%|▏ | 6/435 [00:12<05:47, 1.23it/s]
Epoch 2/50 - Training: 2%|▏ | 7/435 [00:13<04:08, 1.72it/s]
Epoch 2/50 - Training: 2%|▏ | 8/435 [00:13<03:03, 2.33it/s]
Epoch 2/50 - Training: 2%|▏ | 9/435 [00:13<02:20, 3.03it/s]
Epoch 2/50 - Training: 2%|▏ | 10/435 [00:14<05:12, 1.36it/s]
Epoch 2/50 - Training: 3%|▎ | 11/435 [00:25<26:29, 3.75s/it]
Epoch 2/50 - Training: 3%|▎ | 13/435 [00:25<14:22, 2.04s/it]
Epoch 2/50 - Training: 3%|▎ | 15/435 [00:25<08:50, 1.26s/it]
Epoch 2/50 - Training: 4%|▍ | 17/435 [00:26<05:49, 1.19it/s]
Epoch 2/50 - Training: 4%|▍ | 18/435 [00:26<04:45, 1.46it/s]
Epoch 2/50 - Training: 5%|▍ | 20/435 [00:29<07:45, 1.12s/it]
Epoch 2/50 - Training: 5%|▍ | 21/435 [00:36<16:00, 2.32s/it]
Epoch 2/50 - Training: 5%|▌ | 22/435 [00:36<12:28, 1.81s/it]
Epoch 2/50 - Training: 6%|▌ | 24/435 [00:37<07:45, 1.13s/it]
Epoch 2/50 - Training: 6%|▌ | 26/435 [00:37<05:09, 1.32it/s]
Epoch 2/50 - Training: 6%|▋ | 28/435 [00:37<03:35, 1.89it/s]
Epoch 2/50 - Training: 7%|▋ | 29/435 [00:37<03:00, 2.24it/s]
Epoch 2/50 - Training: 7%|▋ | 30/435 [00:41<08:16, 1.23s/it]
Epoch 2/50 - Training: 7%|▋ | 31/435 [00:47<16:12, 2.41s/it]
Epoch 2/50 - Training: 7%|▋ | 32/435 [00:47<12:16, 1.83s/it]
Epoch 2/50 - Training: 8%|▊ | 33/435 [00:47<09:14, 1.38s/it]
Epoch 2/50 - Training: 8%|▊ | 34/435 [00:48<06:56, 1.04s/it]
Epoch 2/50 - Training: 8%|▊ | 35/435 [00:48<05:16, 1.27it/s]
Epoch 2/50 - Training: 8%|▊ | 36/435 [00:48<03:59, 1.66it/s]
Epoch 2/50 - Training: 9%|▊ | 37/435 [00:48<03:06, 2.14it/s]
Epoch 2/50 - Training: 9%|▊ | 38/435 [00:48<02:27, 2.69it/s]
Epoch 2/50 - Training: 9%|▉ | 39/435 [00:48<02:00, 3.28it/s]
Epoch 2/50 - Training: 9%|▉ | 40/435 [00:52<09:15, 1.41s/it]
Epoch 2/50 - Training: 9%|▉ | 41/435 [00:58<17:37, 2.68s/it]
Epoch 2/50 - Training: 10%|▉ | 42/435 [00:58<12:34, 1.92s/it]
Epoch 2/50 - Training: 10%|▉ | 43/435 [00:58<09:04, 1.39s/it]
Epoch 2/50 - Training: 10%|█ | 44/435 [00:58<06:36, 1.01s/it]
Epoch 2/50 - Training: 10%|█ | 45/435 [00:59<04:51, 1.34it/s]
Epoch 2/50 - Training: 11%|█ | 46/435 [00:59<03:37, 1.79it/s]
Epoch 2/50 - Training: 11%|█ | 47/435 [00:59<02:47, 2.32it/s]
Epoch 2/50 - Training: 11%|█ | 48/435 [00:59<02:12, 2.91it/s]
Epoch 2/50 - Training: 11%|█▏ | 49/435 [00:59<01:48, 3.55it/s]
Epoch 2/50 - Training: 11%|█▏ | 50/435 [01:04<10:57, 1.71s/it]
Epoch 2/50 - Training: 12%|█▏ | 51/435 [01:09<17:06, 2.67s/it]
Epoch 2/50 - Training: 12%|█▏ | 52/435 [01:09<12:08, 1.90s/it]
Epoch 2/50 - Training: 12%|█▏ | 53/435 [01:09<08:46, 1.38s/it]
Epoch 2/50 - Training: 12%|█▏ | 54/435 [01:09<06:24, 1.01s/it]
Epoch 2/50 - Training: 13%|█▎ | 55/435 [01:10<04:46, 1.33it/s]
Epoch 2/50 - Training: 13%|█▎ | 56/435 [01:10<03:36, 1.75it/s]
Epoch 2/50 - Training: 13%|█▎ | 57/435 [01:10<02:49, 2.23it/s]
Epoch 2/50 - Training: 13%|█▎ | 58/435 [01:10<02:15, 2.79it/s]
Epoch 2/50 - Training: 14%|█▎ | 59/435 [01:10<01:50, 3.42it/s]
Epoch 2/50 - Training: 14%|█▍ | 60/435 [01:16<12:00, 1.92s/it]
Epoch 2/50 - Training: 14%|█▍ | 61/435 [01:20<15:03, 2.42s/it]
Epoch 2/50 - Training: 14%|█▍ | 62/435 [01:20<10:44, 1.73s/it]
Epoch 2/50 - Training: 15%|█▍ | 64/435 [01:20<06:03, 1.02it/s]
Epoch 2/50 - Training: 15%|█▍ | 65/435 [01:20<04:43, 1.30it/s]
Epoch 2/50 - Training: 15%|█▌ | 66/435 [01:20<03:39, 1.68it/s]
Epoch 2/50 - Training: 15%|█▌ | 67/435 [01:20<02:50, 2.16it/s]
Epoch 2/50 - Training: 16%|█▌ | 68/435 [01:20<02:13, 2.74it/s]
Epoch 2/50 - Training: 16%|█▌ | 69/435 [01:20<01:46, 3.44it/s]
Epoch 2/50 - Training: 16%|█▌ | 70/435 [01:28<14:04, 2.31s/it]
Epoch 2/50 - Training: 16%|█▋ | 71/435 [01:29<12:33, 2.07s/it]
Epoch 2/50 - Training: 17%|█▋ | 72/435 [01:29<09:03, 1.50s/it]
Epoch 2/50 - Training: 17%|█▋ | 74/435 [01:29<05:08, 1.17it/s]
Epoch 2/50 - Training: 17%|█▋ | 76/435 [01:30<03:19, 1.80it/s]
Epoch 2/50 - Training: 18%|█▊ | 77/435 [01:30<02:44, 2.18it/s]
Epoch 2/50 - Training: 18%|█▊ | 79/435 [01:30<01:53, 3.13it/s]
Epoch 2/50 - Training: 18%|█▊ | 80/435 [01:40<14:12, 2.40s/it]
Epoch 2/50 - Training: 19%|█▊ | 81/435 [01:40<10:58, 1.86s/it]
Epoch 2/50 - Training: 19%|█▉ | 82/435 [01:40<08:21, 1.42s/it]
Epoch 2/50 - Training: 19%|█▉ | 83/435 [01:40<06:18, 1.08s/it]
Epoch 2/50 - Training: 19%|█▉ | 84/435 [01:40<04:45, 1.23it/s]
Epoch 2/50 - Training: 20%|█▉ | 86/435 [01:41<02:54, 2.00it/s]
Epoch 2/50 - Training: 20%|██ | 88/435 [01:41<02:00, 2.89it/s]
Epoch 2/50 - Training: 20%|██ | 89/435 [01:41<01:41, 3.41it/s]
Epoch 2/50 - Training: 21%|██ | 90/435 [01:50<13:56, 2.42s/it]
Epoch 2/50 - Training: 21%|██ | 91/435 [01:50<10:34, 1.84s/it]
Epoch 2/50 - Training: 21%|██▏ | 93/435 [01:51<06:18, 1.11s/it]
Epoch 2/50 - Training: 22%|██▏ | 95/435 [01:51<04:05, 1.38it/s]
Epoch 2/50 - Training: 22%|██▏ | 97/435 [01:51<02:48, 2.00it/s]
Epoch 2/50 - Training: 23%|██▎ | 99/435 [01:51<02:01, 2.77it/s]
Epoch 2/50 - Training: 23%|██▎ | 101/435 [01:59<08:52, 1.59s/it]
Epoch 2/50 - Training: 23%|██▎ | 102/435 [01:59<07:18, 1.32s/it]
Epoch 2/50 - Training: 24%|██▎ | 103/435 [02:00<05:52, 1.06s/it]
Epoch 2/50 - Training: 24%|██▍ | 104/435 [02:00<04:38, 1.19it/s]
Epoch 2/50 - Training: 24%|██▍ | 106/435 [02:00<02:59, 1.83it/s]
Epoch 2/50 - Training: 25%|██▍ | 108/435 [02:00<02:05, 2.61it/s]
Epoch 2/50 - Training: 25%|██▌ | 109/435 [02:00<01:47, 3.03it/s]
Epoch 2/50 - Training: 25%|██▌ | 110/435 [02:08<11:13, 2.07s/it]
Epoch 2/50 - Training: 26%|██▌ | 111/435 [02:08<08:35, 1.59s/it]
Epoch 2/50 - Training: 26%|██▌ | 112/435 [02:08<06:31, 1.21s/it]
Epoch 2/50 - Training: 26%|██▌ | 113/435 [02:09<04:54, 1.09it/s]
Epoch 2/50 - Training: 26%|██▌ | 114/435 [02:09<03:42, 1.44it/s]
Epoch 2/50 - Training: 26%|██▋ | 115/435 [02:09<02:48, 1.90it/s]
Epoch 2/50 - Training: 27%|██▋ | 116/435 [02:09<02:09, 2.46it/s]
Epoch 2/50 - Training: 27%|██▋ | 117/435 [02:09<01:41, 3.12it/s]
Epoch 2/50 - Training: 27%|██▋ | 118/435 [02:09<01:21, 3.89it/s]
Epoch 2/50 - Training: 27%|██▋ | 119/435 [02:09<01:08, 4.62it/s]
Epoch 2/50 - Training: 28%|██▊ | 120/435 [02:16<12:01, 2.29s/it]
Epoch 2/50 - Training: 28%|██▊ | 121/435 [02:17<08:35, 1.64s/it]
Epoch 2/50 - Training: 28%|██▊ | 122/435 [02:17<06:12, 1.19s/it]
Epoch 2/50 - Training: 28%|██▊ | 123/435 [02:17<04:31, 1.15it/s]
Epoch 2/50 - Training: 29%|██▊ | 124/435 [02:17<03:20, 1.55it/s]
Epoch 2/50 - Training: 29%|██▊ | 125/435 [02:17<02:30, 2.07it/s]
Epoch 2/50 - Training: 29%|██▉ | 126/435 [02:17<01:54, 2.70it/s]
Epoch 2/50 - Training: 29%|██▉ | 127/435 [02:17<01:29, 3.45it/s]
Epoch 2/50 - Training: 29%|██▉ | 128/435 [02:17<01:13, 4.20it/s]
Epoch 2/50 - Training: 30%|██▉ | 129/435 [02:17<01:01, 5.00it/s]
Epoch 2/50 - Training: 30%|██▉ | 130/435 [02:25<12:10, 2.40s/it]
Epoch 2/50 - Training: 30%|███ | 132/435 [02:25<06:46, 1.34s/it]
Epoch 2/50 - Training: 31%|███ | 133/435 [02:25<05:18, 1.05s/it]
Epoch 2/50 - Training: 31%|███ | 134/435 [02:26<04:06, 1.22it/s]
Epoch 2/50 - Training: 31%|███ | 135/435 [02:26<03:10, 1.58it/s]
Epoch 2/50 - Training: 31%|███▏ | 136/435 [02:26<02:28, 2.02it/s]
Epoch 2/50 - Training: 31%|███▏ | 137/435 [02:26<01:56, 2.57it/s]
Epoch 2/50 - Training: 32%|███▏ | 138/435 [02:26<01:32, 3.22it/s]
Epoch 2/50 - Training: 32%|███▏ | 139/435 [02:26<01:14, 3.95it/s]
Epoch 2/50 - Training: 32%|███▏ | 140/435 [02:34<11:46, 2.39s/it]
Epoch 2/50 - Training: 32%|███▏ | 141/435 [02:34<08:24, 1.72s/it]
Epoch 2/50 - Training: 33%|███▎ | 142/435 [02:34<06:03, 1.24s/it]
Epoch 2/50 - Training: 33%|███▎ | 143/435 [02:34<04:24, 1.10it/s]
Epoch 2/50 - Training: 33%|███▎ | 144/435 [02:34<03:16, 1.48it/s]
Epoch 2/50 - Training: 33%|███▎ | 145/435 [02:34<02:28, 1.95it/s]
Epoch 2/50 - Training: 34%|███▎ | 146/435 [02:34<01:55, 2.51it/s]
Epoch 2/50 - Training: 34%|███▍ | 147/435 [02:35<01:29, 3.20it/s]
Epoch 2/50 - Training: 34%|███▍ | 148/435 [02:35<01:13, 3.92it/s]
Epoch 2/50 - Training: 34%|███▍ | 149/435 [02:35<01:02, 4.59it/s]
Epoch 2/50 - Training: 34%|███▍ | 150/435 [02:43<12:35, 2.65s/it]
Epoch 2/50 - Training: 35%|███▍ | 151/435 [02:43<08:59, 1.90s/it]
Epoch 2/50 - Training: 35%|███▍ | 152/435 [02:43<06:26, 1.36s/it]
Epoch 2/50 - Training: 35%|███▌ | 153/435 [02:44<04:38, 1.01it/s]
Epoch 2/50 - Training: 36%|███▌ | 155/435 [02:44<02:40, 1.74it/s]
Epoch 2/50 - Training: 36%|███▌ | 156/435 [02:44<02:09, 2.16it/s]
Epoch 2/50 - Training: 36%|███▌ | 157/435 [02:44<01:45, 2.64it/s]
Epoch 2/50 - Training: 36%|███▋ | 158/435 [02:44<01:26, 3.21it/s]
Epoch 2/50 - Training: 37%|███▋ | 159/435 [02:44<01:11, 3.85it/s]
Epoch 2/50 - Training: 37%|███▋ | 160/435 [02:52<10:33, 2.30s/it]
Epoch 2/50 - Training: 37%|███▋ | 161/435 [02:52<07:36, 1.66s/it]
Epoch 2/50 - Training: 37%|███▋ | 162/435 [02:52<05:30, 1.21s/it]
Epoch 2/50 - Training: 37%|███▋ | 163/435 [02:52<04:01, 1.13it/s]
Epoch 2/50 - Training: 38%|███▊ | 164/435 [02:52<02:58, 1.51it/s]
Epoch 2/50 - Training: 38%|███▊ | 165/435 [02:52<02:15, 2.00it/s]
Epoch 2/50 - Training: 38%|███▊ | 166/435 [02:52<01:45, 2.56it/s]
Epoch 2/50 - Training: 38%|███▊ | 167/435 [02:52<01:23, 3.19it/s]
Epoch 2/50 - Training: 39%|███▊ | 168/435 [02:53<01:08, 3.92it/s]
Epoch 2/50 - Training: 39%|███▉ | 169/435 [02:53<00:56, 4.68it/s]
Epoch 2/50 - Training: 39%|███▉ | 170/435 [03:01<11:24, 2.58s/it]
Epoch 2/50 - Training: 39%|███▉ | 171/435 [03:01<08:05, 1.84s/it]
Epoch 2/50 - Training: 40%|███▉ | 172/435 [03:01<05:48, 1.32s/it]
Epoch 2/50 - Training: 40%|███▉ | 173/435 [03:01<04:13, 1.03it/s]
Epoch 2/50 - Training: 40%|████ | 174/435 [03:01<03:07, 1.39it/s]
Epoch 2/50 - Training: 40%|████ | 175/435 [03:01<02:21, 1.84it/s]
Epoch 2/50 - Training: 40%|████ | 176/435 [03:02<01:50, 2.35it/s]
Epoch 2/50 - Training: 41%|████ | 177/435 [03:02<01:28, 2.93it/s]
Epoch 2/50 - Training: 41%|████ | 178/435 [03:02<01:11, 3.59it/s]
Epoch 2/50 - Training: 41%|████ | 179/435 [03:02<01:00, 4.22it/s]
Epoch 2/50 - Training: 41%|████▏ | 180/435 [03:10<10:16, 2.42s/it]
Epoch 2/50 - Training: 42%|████▏ | 182/435 [03:10<05:41, 1.35s/it]
Epoch 2/50 - Training: 42%|████▏ | 183/435 [03:10<04:23, 1.04s/it]
Epoch 2/50 - Training: 42%|████▏ | 184/435 [03:10<03:22, 1.24it/s]
Epoch 2/50 - Training: 43%|████▎ | 185/435 [03:10<02:35, 1.61it/s]
Epoch 2/50 - Training: 43%|████▎ | 186/435 [03:10<01:59, 2.08it/s]
Epoch 2/50 - Training: 43%|████▎ | 187/435 [03:10<01:33, 2.65it/s]
Epoch 2/50 - Training: 43%|████▎ | 188/435 [03:10<01:14, 3.31it/s]
Epoch 2/50 - Training: 43%|████▎ | 189/435 [03:11<01:01, 3.97it/s]
Epoch 2/50 - Training: 44%|████▎ | 190/435 [03:18<10:02, 2.46s/it]
Epoch 2/50 - Training: 44%|████▍ | 192/435 [03:19<05:36, 1.39s/it]
Epoch 2/50 - Training: 44%|████▍ | 193/435 [03:19<04:21, 1.08s/it]
Epoch 2/50 - Training: 45%|████▍ | 194/435 [03:19<03:21, 1.20it/s]
Epoch 2/50 - Training: 45%|████▍ | 195/435 [03:19<02:35, 1.54it/s]
Epoch 2/50 - Training: 45%|████▌ | 196/435 [03:19<02:01, 1.96it/s]
Epoch 2/50 - Training: 45%|████▌ | 197/435 [03:19<01:35, 2.49it/s]
Epoch 2/50 - Training: 46%|████▌ | 198/435 [03:19<01:17, 3.06it/s]
Epoch 2/50 - Training: 46%|████▌ | 199/435 [03:20<01:03, 3.69it/s]
Epoch 2/50 - Training: 46%|████▌ | 200/435 [03:28<10:03, 2.57s/it]
Epoch 2/50 - Training: 46%|████▋ | 202/435 [03:28<05:35, 1.44s/it]
Epoch 2/50 - Training: 47%|████▋ | 203/435 [03:28<04:18, 1.11s/it]
Epoch 2/50 - Training: 47%|████▋ | 204/435 [03:28<03:17, 1.17it/s]
Epoch 2/50 - Training: 47%|████▋ | 205/435 [03:28<02:30, 1.53it/s]
Epoch 2/50 - Training: 47%|████▋ | 206/435 [03:28<01:55, 1.98it/s]
Epoch 2/50 - Training: 48%|████▊ | 207/435 [03:28<01:29, 2.54it/s]
Epoch 2/50 - Training: 48%|████▊ | 208/435 [03:29<01:11, 3.18it/s]
Epoch 2/50 - Training: 48%|████▊ | 209/435 [03:29<00:57, 3.90it/s]
Epoch 2/50 - Training: 48%|████▊ | 210/435 [03:36<09:05, 2.42s/it]
Epoch 2/50 - Training: 49%|████▊ | 212/435 [03:36<05:03, 1.36s/it]
Epoch 2/50 - Training: 49%|████▉ | 213/435 [03:37<03:55, 1.06s/it]
Epoch 2/50 - Training: 49%|████▉ | 214/435 [03:37<03:00, 1.22it/s]
Epoch 2/50 - Training: 49%|████▉ | 215/435 [03:37<02:19, 1.58it/s]
Epoch 2/50 - Training: 50%|████▉ | 216/435 [03:37<01:48, 2.02it/s]
Epoch 2/50 - Training: 50%|████▉ | 217/435 [03:37<01:24, 2.57it/s]
Epoch 2/50 - Training: 50%|█████ | 218/435 [03:37<01:07, 3.20it/s]
Epoch 2/50 - Training: 50%|█████ | 219/435 [03:37<00:57, 3.76it/s]
Epoch 2/50 - Training: 51%|█████ | 220/435 [03:45<08:55, 2.49s/it]
Epoch 2/50 - Training: 51%|█████ | 222/435 [03:45<04:57, 1.40s/it]
Epoch 2/50 - Training: 51%|█████▏ | 223/435 [03:46<03:50, 1.09s/it]
Epoch 2/50 - Training: 51%|█████▏ | 224/435 [03:46<02:57, 1.19it/s]
Epoch 2/50 - Training: 52%|█████▏ | 225/435 [03:46<02:16, 1.54it/s]
Epoch 2/50 - Training: 52%|█████▏ | 226/435 [03:46<01:45, 1.99it/s]
Epoch 2/50 - Training: 52%|█████▏ | 227/435 [03:46<01:22, 2.53it/s]
Epoch 2/50 - Training: 52%|█████▏ | 228/435 [03:46<01:05, 3.14it/s]
Epoch 2/50 - Training: 53%|█████▎ | 229/435 [03:46<00:54, 3.79it/s]
Epoch 2/50 - Training: 53%|█████▎ | 230/435 [03:54<07:59, 2.34s/it]
Epoch 2/50 - Training: 53%|█████▎ | 232/435 [03:54<04:26, 1.31s/it]
Epoch 2/50 - Training: 54%|█████▎ | 233/435 [03:54<03:25, 1.02s/it]
Epoch 2/50 - Training: 54%|█████▍ | 235/435 [03:54<02:07, 1.57it/s]
Epoch 2/50 - Training: 54%|█████▍ | 237/435 [03:54<01:26, 2.28it/s]
Epoch 2/50 - Training: 55%|█████▍ | 238/435 [03:54<01:13, 2.67it/s]
Epoch 2/50 - Training: 55%|█████▍ | 239/435 [03:55<01:02, 3.16it/s]
Epoch 2/50 - Training: 55%|█████▌ | 240/435 [04:03<07:11, 2.21s/it]
Epoch 2/50 - Training: 56%|█████▌ | 242/435 [04:03<04:17, 1.33s/it]
Epoch 2/50 - Training: 56%|█████▌ | 243/435 [04:03<03:22, 1.05s/it]
Epoch 2/50 - Training: 56%|█████▌ | 244/435 [04:03<02:37, 1.21it/s]
Epoch 2/50 - Training: 56%|█████▋ | 245/435 [04:03<02:02, 1.55it/s]
Epoch 2/50 - Training: 57%|█████▋ | 246/435 [04:03<01:34, 1.99it/s]
Epoch 2/50 - Training: 57%|█████▋ | 247/435 [04:03<01:14, 2.52it/s]
Epoch 2/50 - Training: 57%|█████▋ | 248/435 [04:03<00:59, 3.15it/s]
Epoch 2/50 - Training: 57%|█████▋ | 249/435 [04:04<00:49, 3.79it/s]
Epoch 2/50 - Training: 57%|█████▋ | 250/435 [04:11<07:03, 2.29s/it]
Epoch 2/50 - Training: 58%|█████▊ | 252/435 [04:11<03:56, 1.29s/it]
Epoch 2/50 - Training: 58%|█████▊ | 253/435 [04:11<03:04, 1.01s/it]
Epoch 2/50 - Training: 58%|█████▊ | 254/435 [04:11<02:21, 1.28it/s]
Epoch 2/50 - Training: 59%|█████▊ | 255/435 [04:11<01:49, 1.65it/s]
Epoch 2/50 - Training: 59%|█████▉ | 256/435 [04:12<01:25, 2.11it/s]
Epoch 2/50 - Training: 59%|█████▉ | 257/435 [04:12<01:07, 2.64it/s]
Epoch 2/50 - Training: 59%|█████▉ | 258/435 [04:12<00:54, 3.27it/s]
Epoch 2/50 - Training: 60%|█████▉ | 259/435 [04:12<00:44, 3.94it/s]
Epoch 2/50 - Training: 60%|█████▉ | 260/435 [04:19<06:28, 2.22s/it]
Epoch 2/50 - Training: 60%|██████ | 262/435 [04:19<03:35, 1.25s/it]
Epoch 2/50 - Training: 60%|██████ | 263/435 [04:19<02:46, 1.03it/s]
Epoch 2/50 - Training: 61%|██████ | 264/435 [04:19<02:07, 1.34it/s]
Epoch 2/50 - Training: 61%|██████ | 265/435 [04:19<01:38, 1.73it/s]
Epoch 2/50 - Training: 61%|██████ | 266/435 [04:19<01:16, 2.22it/s]
Epoch 2/50 - Training: 61%|██████▏ | 267/435 [04:20<00:59, 2.81it/s]
Epoch 2/50 - Training: 62%|██████▏ | 268/435 [04:20<00:48, 3.47it/s]
Epoch 2/50 - Training: 62%|██████▏ | 269/435 [04:20<00:39, 4.18it/s]
Epoch 2/50 - Training: 62%|██████▏ | 270/435 [04:27<06:29, 2.36s/it]
Epoch 2/50 - Training: 63%|██████▎ | 272/435 [04:27<03:35, 1.32s/it]
Epoch 2/50 - Training: 63%|██████▎ | 273/435 [04:28<02:46, 1.03s/it]
Epoch 2/50 - Training: 63%|██████▎ | 274/435 [04:28<02:08, 1.25it/s]
Epoch 2/50 - Training: 63%|██████▎ | 275/435 [04:28<01:39, 1.61it/s]
Epoch 2/50 - Training: 63%|██████▎ | 276/435 [04:28<01:17, 2.04it/s]
Epoch 2/50 - Training: 64%|██████▎ | 277/435 [04:28<01:02, 2.52it/s]
Epoch 2/50 - Training: 64%|██████▍ | 278/435 [04:28<00:50, 3.10it/s]
Epoch 2/50 - Training: 64%|██████▍ | 279/435 [04:28<00:41, 3.72it/s]
Epoch 2/50 - Training: 64%|██████▍ | 280/435 [04:36<05:53, 2.28s/it]
Epoch 2/50 - Training: 65%|██████▍ | 282/435 [04:36<03:16, 1.28s/it]
Epoch 2/50 - Training: 65%|██████▌ | 283/435 [04:36<02:31, 1.00it/s]
Epoch 2/50 - Training: 65%|██████▌ | 284/435 [04:36<01:55, 1.31it/s]
Epoch 2/50 - Training: 66%|██████▌ | 285/435 [04:36<01:28, 1.70it/s]
Epoch 2/50 - Training: 66%|██████▌ | 286/435 [04:36<01:08, 2.18it/s]
Epoch 2/50 - Training: 66%|██████▌ | 287/435 [04:36<00:54, 2.73it/s]
Epoch 2/50 - Training: 66%|██████▌ | 288/435 [04:36<00:43, 3.37it/s]
Epoch 2/50 - Training: 66%|██████▋ | 289/435 [04:37<00:35, 4.14it/s]
Epoch 2/50 - Training: 67%|██████▋ | 290/435 [04:43<05:08, 2.12s/it]
Epoch 2/50 - Training: 67%|██████▋ | 292/435 [04:43<02:50, 1.19s/it]
Epoch 2/50 - Training: 67%|██████▋ | 293/435 [04:44<02:11, 1.08it/s]
Epoch 2/50 - Training: 68%|██████▊ | 294/435 [04:44<01:41, 1.39it/s]
Epoch 2/50 - Training: 68%|██████▊ | 295/435 [04:44<01:18, 1.78it/s]
Epoch 2/50 - Training: 68%|██████▊ | 296/435 [04:44<01:01, 2.28it/s]
Epoch 2/50 - Training: 68%|██████▊ | 297/435 [04:44<00:48, 2.85it/s]
Epoch 2/50 - Training: 69%|██████▊ | 298/435 [04:44<00:40, 3.41it/s]
Epoch 2/50 - Training: 69%|██████▊ | 299/435 [04:44<00:33, 4.06it/s]
Epoch 2/50 - Training: 69%|██████▉ | 300/435 [04:51<04:57, 2.20s/it]
Epoch 2/50 - Training: 69%|██████▉ | 302/435 [04:51<02:44, 1.23s/it]
Epoch 2/50 - Training: 70%|██████▉ | 303/435 [04:52<02:07, 1.03it/s]
Epoch 2/50 - Training: 70%|██████▉ | 304/435 [04:52<01:38, 1.33it/s]
Epoch 2/50 - Training: 70%|███████ | 305/435 [04:52<01:15, 1.72it/s]
Epoch 2/50 - Training: 70%|███████ | 306/435 [04:52<00:58, 2.20it/s]
Epoch 2/50 - Training: 71%|███████ | 307/435 [04:52<00:46, 2.76it/s]
Epoch 2/50 - Training: 71%|███████ | 308/435 [04:52<00:37, 3.41it/s]
Epoch 2/50 - Training: 71%|███████ | 309/435 [04:52<00:30, 4.08it/s]
Epoch 2/50 - Training: 71%|███████▏ | 310/435 [04:59<04:23, 2.11s/it]
Epoch 2/50 - Training: 72%|███████▏ | 312/435 [04:59<02:25, 1.19s/it]
Epoch 2/50 - Training: 72%|███████▏ | 313/435 [04:59<01:53, 1.08it/s]
Epoch 2/50 - Training: 72%|███████▏ | 314/435 [04:59<01:27, 1.39it/s]
Epoch 2/50 - Training: 72%|███████▏ | 315/435 [04:59<01:06, 1.79it/s]
Epoch 2/50 - Training: 73%|███████▎ | 316/435 [05:00<00:51, 2.30it/s]
Epoch 2/50 - Training: 73%|███████▎ | 317/435 [05:00<00:40, 2.91it/s]
Epoch 2/50 - Training: 73%|███████▎ | 318/435 [05:00<00:32, 3.60it/s]
Epoch 2/50 - Training: 73%|███████▎ | 319/435 [05:00<00:27, 4.29it/s]
Epoch 2/50 - Training: 74%|███████▎ | 320/435 [05:07<04:06, 2.15s/it]
Epoch 2/50 - Training: 74%|███████▍ | 322/435 [05:07<02:16, 1.21s/it]
Epoch 2/50 - Training: 74%|███████▍ | 323/435 [05:07<01:46, 1.05it/s]
Epoch 2/50 - Training: 74%|███████▍ | 324/435 [05:07<01:22, 1.35it/s]
Epoch 2/50 - Training: 75%|███████▍ | 325/435 [05:07<01:03, 1.73it/s]
Epoch 2/50 - Training: 75%|███████▍ | 326/435 [05:07<00:49, 2.19it/s]
Epoch 2/50 - Training: 75%|███████▌ | 327/435 [05:08<00:39, 2.72it/s]
Epoch 2/50 - Training: 75%|███████▌ | 328/435 [05:08<00:32, 3.32it/s]
Epoch 2/50 - Training: 76%|███████▌ | 329/435 [05:08<00:27, 3.89it/s]
Epoch 2/50 - Training: 76%|███████▌ | 330/435 [05:14<03:18, 1.90s/it]
Epoch 2/50 - Training: 76%|███████▋ | 332/435 [05:14<01:51, 1.08s/it]
Epoch 2/50 - Training: 77%|███████▋ | 333/435 [05:14<01:26, 1.19it/s]
Epoch 2/50 - Training: 77%|███████▋ | 334/435 [05:14<01:06, 1.53it/s]
Epoch 2/50 - Training: 77%|███████▋ | 335/435 [05:14<00:50, 1.97it/s]
Epoch 2/50 - Training: 77%|███████▋ | 336/435 [05:14<00:39, 2.50it/s]
Epoch 2/50 - Training: 77%|███████▋ | 337/435 [05:14<00:31, 3.13it/s]
Epoch 2/50 - Training: 78%|███████▊ | 338/435 [05:15<00:25, 3.86it/s]
Epoch 2/50 - Training: 78%|███████▊ | 339/435 [05:15<00:21, 4.50it/s]
Epoch 2/50 - Training: 78%|███████▊ | 340/435 [05:21<03:18, 2.09s/it]
Epoch 2/50 - Training: 79%|███████▊ | 342/435 [05:21<01:49, 1.17s/it]
Epoch 2/50 - Training: 79%|███████▉ | 343/435 [05:22<01:24, 1.09it/s]
Epoch 2/50 - Training: 79%|███████▉ | 344/435 [05:22<01:04, 1.41it/s]
Epoch 2/50 - Training: 79%|███████▉ | 345/435 [05:22<00:49, 1.82it/s]
Epoch 2/50 - Training: 80%|███████▉ | 346/435 [05:22<00:38, 2.34it/s]
Epoch 2/50 - Training: 80%|███████▉ | 347/435 [05:22<00:30, 2.92it/s]
Epoch 2/50 - Training: 80%|████████ | 348/435 [05:22<00:23, 3.64it/s]
Epoch 2/50 - Training: 80%|████████ | 349/435 [05:22<00:19, 4.38it/s]
Epoch 2/50 - Training: 80%|████████ | 350/435 [05:30<03:24, 2.40s/it]
Epoch 2/50 - Training: 81%|████████ | 352/435 [05:30<01:51, 1.34s/it]
Epoch 2/50 - Training: 81%|████████ | 353/435 [05:30<01:25, 1.05s/it]
Epoch 2/50 - Training: 81%|████████▏ | 354/435 [05:30<01:05, 1.25it/s]
Epoch 2/50 - Training: 82%|████████▏ | 355/435 [05:30<00:49, 1.62it/s]
Epoch 2/50 - Training: 82%|████████▏ | 356/435 [05:31<00:38, 2.08it/s]
Epoch 2/50 - Training: 82%|████████▏ | 357/435 [05:31<00:29, 2.63it/s]
Epoch 2/50 - Training: 82%|████████▏ | 358/435 [05:31<00:23, 3.25it/s]
Epoch 2/50 - Training: 83%|████████▎ | 360/435 [05:39<02:27, 1.97s/it]
Epoch 2/50 - Training: 83%|████████▎ | 362/435 [05:39<01:29, 1.23s/it]
Epoch 2/50 - Training: 83%|████████▎ | 363/435 [05:39<01:10, 1.01it/s]
Epoch 2/50 - Training: 84%|████████▎ | 364/435 [05:39<00:55, 1.29it/s]
Epoch 2/50 - Training: 84%|████████▍ | 366/435 [05:39<00:34, 1.98it/s]
Epoch 2/50 - Training: 84%|████████▍ | 367/435 [05:40<00:28, 2.36it/s]
Epoch 2/50 - Training: 85%|████████▍ | 368/435 [05:40<00:23, 2.82it/s]
Epoch 2/50 - Training: 85%|████████▍ | 369/435 [05:40<00:19, 3.35it/s]
Epoch 2/50 - Training: 85%|████████▌ | 370/435 [05:48<02:34, 2.37s/it]
Epoch 2/50 - Training: 86%|████████▌ | 372/435 [05:48<01:26, 1.38s/it]
Epoch 2/50 - Training: 86%|████████▌ | 373/435 [05:48<01:06, 1.08s/it]
Epoch 2/50 - Training: 86%|████████▌ | 374/435 [05:48<00:50, 1.20it/s]
Epoch 2/50 - Training: 86%|████████▌ | 375/435 [05:48<00:38, 1.56it/s]
Epoch 2/50 - Training: 86%|████████▋ | 376/435 [05:48<00:29, 2.02it/s]
Epoch 2/50 - Training: 87%|████████▋ | 377/435 [05:49<00:22, 2.60it/s]
Epoch 2/50 - Training: 87%|████████▋ | 378/435 [05:49<00:17, 3.30it/s]
Epoch 2/50 - Training: 87%|████████▋ | 379/435 [05:49<00:14, 3.99it/s]
Epoch 2/50 - Training: 87%|████████▋ | 380/435 [05:57<02:30, 2.73s/it]
Epoch 2/50 - Training: 88%|████████▊ | 382/435 [05:58<01:20, 1.53s/it]
Epoch 2/50 - Training: 88%|████████▊ | 383/435 [05:58<01:01, 1.18s/it]
Epoch 2/50 - Training: 88%|████████▊ | 384/435 [05:58<00:45, 1.11it/s]
Epoch 2/50 - Training: 89%|████████▊ | 385/435 [05:58<00:34, 1.46it/s]
Epoch 2/50 - Training: 89%|████████▊ | 386/435 [05:58<00:25, 1.90it/s]
Epoch 2/50 - Training: 89%|████████▉ | 387/435 [05:58<00:20, 2.39it/s]
Epoch 2/50 - Training: 89%|████████▉ | 388/435 [05:58<00:15, 3.00it/s]
Epoch 2/50 - Training: 89%|████████▉ | 389/435 [05:58<00:12, 3.68it/s]
Epoch 2/50 - Training: 90%|████████▉ | 390/435 [06:06<01:54, 2.55s/it]
Epoch 2/50 - Training: 90%|█████████ | 392/435 [06:07<01:01, 1.42s/it]
Epoch 2/50 - Training: 91%|█████████ | 394/435 [06:07<00:36, 1.11it/s]
Epoch 2/50 - Training: 91%|█████████ | 396/435 [06:07<00:23, 1.64it/s]
Epoch 2/50 - Training: 91%|█████████▏| 397/435 [06:07<00:19, 1.97it/s]
Epoch 2/50 - Training: 92%|█████████▏| 399/435 [06:07<00:12, 2.79it/s]
Epoch 2/50 - Training: 92%|█████████▏| 400/435 [06:15<01:05, 1.86s/it]
Epoch 2/50 - Training: 92%|█████████▏| 402/435 [06:15<00:39, 1.19s/it]
Epoch 2/50 - Training: 93%|█████████▎| 404/435 [06:15<00:25, 1.23it/s]
Epoch 2/50 - Training: 93%|█████████▎| 405/435 [06:15<00:20, 1.49it/s]
Epoch 2/50 - Training: 93%|█████████▎| 406/435 [06:15<00:15, 1.82it/s]
Epoch 2/50 - Training: 94%|█████████▎| 407/435 [06:16<00:12, 2.24it/s]
Epoch 2/50 - Training: 94%|█████████▍| 408/435 [06:16<00:09, 2.76it/s]
Epoch 2/50 - Training: 94%|█████████▍| 409/435 [06:16<00:07, 3.41it/s]
Epoch 2/50 - Training: 94%|█████████▍| 410/435 [06:23<00:55, 2.21s/it]
Epoch 2/50 - Training: 95%|█████████▍| 412/435 [06:23<00:29, 1.27s/it]
Epoch 2/50 - Training: 95%|█████████▌| 414/435 [06:23<00:17, 1.22it/s]
Epoch 2/50 - Training: 95%|█████████▌| 415/435 [06:24<00:13, 1.49it/s]
Epoch 2/50 - Training: 96%|█████████▌| 416/435 [06:24<00:10, 1.85it/s]
Epoch 2/50 - Training: 96%|█████████▌| 417/435 [06:24<00:07, 2.29it/s]
Epoch 2/50 - Training: 96%|█████████▌| 418/435 [06:24<00:05, 2.84it/s]
Epoch 2/50 - Training: 97%|█████████▋| 420/435 [06:29<00:20, 1.34s/it]
Epoch 2/50 - Training: 97%|█████████▋| 422/435 [06:29<00:11, 1.17it/s]
Epoch 2/50 - Training: 97%|█████████▋| 424/435 [06:30<00:06, 1.70it/s]
Epoch 2/50 - Training: 98%|█████████▊| 425/435 [06:30<00:04, 2.02it/s]
Epoch 2/50 - Training: 98%|█████████▊| 426/435 [06:30<00:03, 2.46it/s]
Epoch 2/50 - Training: 98%|█████████▊| 427/435 [06:30<00:02, 3.00it/s]
Epoch 2/50 - Training: 98%|█████████▊| 428/435 [06:30<00:01, 3.61it/s]
Epoch 2/50 - Training: 99%|█████████▉| 430/435 [06:34<00:04, 1.06it/s]
Epoch 2/50 - Training: 99%|█████████▉| 432/435 [06:34<00:01, 1.60it/s]
Epoch 2/50 - Training: 100%|█████████▉| 434/435 [06:34<00:00, 2.28it/s]
Epoch 2/50 - Training: 100%|██████████| 435/435 [06:34<00:00, 1.10it/s]
Epoch 2/50 - Validation: 0%| | 0/48 [00:00<?, ?it/s]
Epoch 2/50 - Validation: 2%|▏ | 1/48 [00:04<03:43, 4.75s/it]
Epoch 2/50 - Validation: 4%|▍ | 2/48 [00:09<03:29, 4.55s/it]
Epoch 2/50 - Validation: 6%|▋ | 3/48 [00:13<03:28, 4.64s/it]
Epoch 2/50 - Validation: 8%|▊ | 4/48 [00:18<03:17, 4.48s/it]
Epoch 2/50 - Validation: 10%|█ | 5/48 [00:22<03:04, 4.29s/it]
Epoch 2/50 - Validation: 12%|█▎ | 6/48 [00:26<02:59, 4.27s/it]
Epoch 2/50 - Validation: 15%|█▍ | 7/48 [00:30<02:50, 4.15s/it]
Epoch 2/50 - Validation: 17%|█▋ | 8/48 [00:34<02:45, 4.15s/it]
Epoch 2/50 - Validation: 19%|█▉ | 9/48 [00:38<02:36, 4.01s/it]
Epoch 2/50 - Validation: 21%|██ | 10/48 [00:41<02:30, 3.96s/it]
Epoch 2/50 - Validation: 23%|██▎ | 11/48 [00:45<02:27, 3.98s/it]
Epoch 2/50 - Validation: 25%|██▌ | 12/48 [00:49<02:18, 3.83s/it]
Epoch 2/50 - Validation: 27%|██▋ | 13/48 [00:53<02:13, 3.82s/it]
Epoch 2/50 - Validation: 29%|██▉ | 14/48 [00:56<02:08, 3.79s/it]
Epoch 2/50 - Validation: 31%|███▏ | 15/48 [01:00<02:01, 3.70s/it]
Epoch 2/50 - Validation: 33%|███▎ | 16/48 [01:04<01:58, 3.70s/it]
Epoch 2/50 - Validation: 35%|███▌ | 17/48 [01:07<01:53, 3.66s/it]
Epoch 2/50 - Validation: 38%|███▊ | 18/48 [01:10<01:45, 3.52s/it]
Epoch 2/50 - Validation: 40%|███▉ | 19/48 [01:14<01:40, 3.46s/it]
Epoch 2/50 - Validation: 42%|████▏ | 20/48 [01:17<01:35, 3.39s/it]
Epoch 2/50 - Validation: 44%|████▍ | 21/48 [01:20<01:31, 3.39s/it]
Epoch 2/50 - Validation: 46%|████▌ | 22/48 [01:24<01:26, 3.34s/it]
Epoch 2/50 - Validation: 48%|████▊ | 23/48 [01:27<01:22, 3.29s/it]
Epoch 2/50 - Validation: 50%|█████ | 24/48 [01:30<01:19, 3.32s/it]
Epoch 2/50 - Validation: 52%|█████▏ | 25/48 [01:33<01:15, 3.27s/it]
Epoch 2/50 - Validation: 54%|█████▍ | 26/48 [01:36<01:11, 3.23s/it]
Epoch 2/50 - Validation: 56%|█████▋ | 27/48 [01:40<01:07, 3.22s/it]
Epoch 2/50 - Validation: 58%|█████▊ | 28/48 [01:43<01:04, 3.23s/it]
Epoch 2/50 - Validation: 60%|██████ | 29/48 [01:46<01:01, 3.24s/it]
Epoch 2/50 - Validation: 62%|██████▎ | 30/48 [01:49<00:57, 3.20s/it]
Epoch 2/50 - Validation: 65%|██████▍ | 31/48 [01:53<00:55, 3.24s/it]
Epoch 2/50 - Validation: 67%|██████▋ | 32/48 [01:56<00:51, 3.23s/it]
Epoch 2/50 - Validation: 69%|██████▉ | 33/48 [01:59<00:48, 3.21s/it]
Epoch 2/50 - Validation: 71%|███████ | 34/48 [02:02<00:44, 3.21s/it]
Epoch 2/50 - Validation: 73%|███████▎ | 35/48 [02:06<00:43, 3.33s/it]
Epoch 2/50 - Validation: 75%|███████▌ | 36/48 [02:09<00:39, 3.27s/it]
Epoch 2/50 - Validation: 77%|███████▋ | 37/48 [02:12<00:35, 3.23s/it]
Epoch 2/50 - Validation: 79%|███████▉ | 38/48 [02:16<00:33, 3.32s/it]
Epoch 2/50 - Validation: 81%|████████▏ | 39/48 [02:19<00:28, 3.21s/it]
Epoch 2/50 - Validation: 83%|████████▎ | 40/48 [02:22<00:25, 3.16s/it]
Epoch 2/50 - Validation: 85%|████████▌ | 41/48 [02:25<00:22, 3.18s/it]
Epoch 2/50 - Validation: 88%|████████▊ | 42/48 [02:28<00:19, 3.19s/it]
Epoch 2/50 - Validation: 90%|████████▉ | 43/48 [02:31<00:15, 3.18s/it]
Epoch 2/50 - Validation: 92%|█████████▏| 44/48 [02:34<00:12, 3.19s/it]
Epoch 2/50 - Validation: 94%|█████████▍| 45/48 [02:38<00:09, 3.23s/it]
Epoch 2/50 - Validation: 96%|█████████▌| 46/48 [02:41<00:06, 3.24s/it]
Epoch 2/50 - Validation: 98%|█████████▊| 47/48 [02:44<00:03, 3.18s/it]
Epoch 2/50 - Validation: 100%|██████████| 48/48 [02:45<00:00, 2.42s/it]
Epoch 2/50 - Validation: 100%|██████████| 48/48 [02:45<00:00, 3.44s/it]
Epoch 3/50 - Training: 0%| | 0/435 [00:00<?, ?it/s]
Epoch 3/50 - Training: 0%| | 1/435 [00:14<1:48:09, 14.95s/it]
Epoch 3/50 - Training: 0%| | 2/435 [00:15<44:52, 6.22s/it]
Epoch 3/50 - Training: 1%| | 4/435 [00:15<17:07, 2.38s/it]
Epoch 3/50 - Training: 1%|▏ | 6/435 [00:15<09:19, 1.30s/it]
Epoch 3/50 - Training: 2%|▏ | 7/435 [00:15<07:08, 1.00s/it]
Epoch 3/50 - Training: 2%|▏ | 8/435 [00:15<05:25, 1.31it/s]
Epoch 3/50 - Training: 2%|▏ | 10/435 [00:15<03:21, 2.11it/s]
Epoch 3/50 - Training: 2%|▏ | 10/435 [00:26<03:21, 2.11it/s]
Epoch 3/50 - Training: 3%|▎ | 11/435 [00:26<20:33, 2.91s/it]
Epoch 3/50 - Training: 3%|▎ | 12/435 [00:26<15:38, 2.22s/it]
Epoch 3/50 - Training: 3%|▎ | 13/435 [00:26<11:42, 1.67s/it]
Epoch 3/50 - Training: 3%|▎ | 14/435 [00:27<08:42, 1.24s/it]
Epoch 3/50 - Training: 3%|▎ | 15/435 [00:27<06:33, 1.07it/s]
Epoch 3/50 - Training: 4%|▎ | 16/435 [00:27<04:59, 1.40it/s]
Epoch 3/50 - Training: 4%|▍ | 17/435 [00:27<03:51, 1.81it/s]
Epoch 3/50 - Training: 4%|▍ | 18/435 [00:27<03:02, 2.28it/s]
Epoch 3/50 - Training: 4%|▍ | 19/435 [00:27<02:27, 2.82it/s]
Epoch 3/50 - Training: 5%|▍ | 20/435 [00:28<02:03, 3.36it/s]
Epoch 3/50 - Training: 5%|▍ | 21/435 [00:36<17:53, 2.59s/it]
Epoch 3/50 - Training: 5%|▌ | 22/435 [00:36<13:43, 1.99s/it]
Epoch 3/50 - Training: 5%|▌ | 23/435 [00:37<10:37, 1.55s/it]
Epoch 3/50 - Training: 6%|▌ | 24/435 [00:37<07:40, 1.12s/it]
Epoch 3/50 - Training: 6%|▌ | 25/435 [00:37<05:38, 1.21it/s]
Epoch 3/50 - Training: 6%|▌ | 26/435 [00:37<04:12, 1.62it/s]
Epoch 3/50 - Training: 6%|▌ | 27/435 [00:37<03:12, 2.12it/s]
Epoch 3/50 - Training: 6%|▋ | 28/435 [00:37<02:32, 2.67it/s]
Epoch 3/50 - Training: 7%|▋ | 29/435 [00:37<02:03, 3.28it/s]
Epoch 3/50 - Training: 7%|▋ | 30/435 [00:38<01:43, 3.91it/s]
Epoch 3/50 - Training: 7%|▋ | 31/435 [00:45<16:30, 2.45s/it]
Epoch 3/50 - Training: 7%|▋ | 32/435 [00:46<13:26, 2.00s/it]
Epoch 3/50 - Training: 8%|▊ | 33/435 [00:47<10:16, 1.53s/it]
Epoch 3/50 - Training: 8%|▊ | 34/435 [00:47<07:25, 1.11s/it]
Epoch 3/50 - Training: 8%|▊ | 35/435 [00:47<05:28, 1.22it/s]
Epoch 3/50 - Training: 8%|▊ | 36/435 [00:47<04:07, 1.61it/s]
Epoch 3/50 - Training: 9%|▊ | 37/435 [00:47<03:11, 2.07it/s]
Epoch 3/50 - Training: 9%|▊ | 38/435 [00:47<02:32, 2.60it/s]
Epoch 3/50 - Training: 9%|▉ | 39/435 [00:47<02:05, 3.16it/s]
Epoch 3/50 - Training: 9%|▉ | 40/435 [00:48<01:44, 3.79it/s]
Epoch 3/50 - Training: 9%|▉ | 41/435 [00:56<17:38, 2.69s/it]slurmstepd: error: *** REASON: burst_buffer/lua: Stage-out in progress ***
slurmstepd: error: *** JOB 541349 ON ip-10-0-135-126 CANCELLED AT 2024-11-27T02:28:13 ***
slurmstepd: error: *** REASON: burst_buffer/lua: Stage-out in progress ***
Epoch 3/50 - Training: 10%|▉ | 42/435 [00:57<14:09, 2.16s/it]
Epoch 3/50 - Training: 10%|▉ | 43/435 [00:57<10:22, 1.59s/it]
Epoch 3/50 - Training: 10%|█ | 44/435 [00:57<07:27, 1.15s/it]
Epoch 3/50 - Training: 10%|█ | 45/435 [00:57<05:27, 1.19it/s]
Epoch 3/50 - Training: 11%|█ | 46/435 [00:57<04:04, 1.59it/s]
Epoch 3/50 - Training: 11%|█ | 47/435 [00:58<03:06, 2.08it/s] |