[15:46:37] Upload loop started. Checking /data/v3_checkpoints every 300s. [15:46:37] Uploading step 45000... Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...nts/optim_045000_rank5.pt: 0%| | 587kB / 2.49GB  ...nts/optim_045000_rank5.pt: 0%| | 587kB / 2.49GB  Processing Files (0 / 1) : 0%| | 587kB / 22.2GB, 587kB/s New Data Upload : 0%| | 587kB / 604MB, 587kB/s  ...nts/optim_045000_rank5.pt: 0%| | 587kB / 2.49GB  ...nts/optim_045000_rank5.pt: 0%| | 1.17MB / 2.49GB  Processing Files (0 / 1) : 0%| | 1.17MB / 22.2GB, 839kB/s New Data Upload : 0%| | 1.17MB / 604MB, 839kB/s  ...nts/optim_045000_rank5.pt: 0%| | 4.11MB / 2.49GB  Processing Files (0 / 1) : 0%| | 4.11MB / 22.2GB, 2.57MB/s New Data Upload : 1%| | 4.11MB / 604MB, 2.57MB/s  ...nts/optim_045000_rank5.pt: 0%| | 8.22MB / 2.49GB  Processing Files (0 / 1) : 0%| | 8.22MB / 22.2GB, 4.57MB/s New Data Upload : 1%|▏ | 8.22MB / 604MB, 4.57MB/s  ...nts/optim_045000_rank5.pt: 1%| | 12.9MB / 2.49GB  Processing Files (0 / 1) : 0%| | 12.9MB / 22.2GB, 6.46MB/s New Data Upload : 2%|▏ | 12.9MB / 604MB, 6.46MB/s  ...nts/optim_045000_rank5.pt: 1%| | 15.3MB / 2.49GB  Processing Files (0 / 1) : 0%| | 15.3MB / 22.2GB, 6.94MB/s New Data Upload : 3%|▎ | 15.3MB / 604MB, 6.94MB/s  ...nts/optim_045000_rank5.pt: 1%| | 18.2MB / 2.49GB  Processing Files (0 / 1) : 0%| | 18.2MB / 22.2GB, 7.58MB/s New Data Upload : 3%|▎ | 18.2MB / 604MB, 7.58MB/s  ...nts/optim_045000_rank5.pt: 1%| | 21.7MB / 2.49GB  Processing Files (0 / 1) : 0%| | 21.7MB / 22.2GB, 8.36MB/s New Data Upload : 4%|▎ | 21.7MB / 604MB, 8.36MB/s  ...nts/optim_045000_rank5.pt: 1%| | 24.7MB / 2.49GB  Processing Files (0 / 1) : 0%| | 24.7MB / 22.2GB, 8.81MB/s New Data Upload : 4%|▍ | 24.7MB / 604MB, 8.81MB/s  ...nts/optim_045000_rank5.pt: 1%| | 28.8MB / 2.49GB  Processing Files (0 / 1) : 0%| | 28.8MB / 22.2GB, 9.59MB/s New Data Upload : 5%|▍ | 28.8MB / 604MB, 9.59MB/s  ...nts/optim_045000_rank5.pt: 1%|▏ | 31.1MB / 2.49GB  Processing Files (0 / 1) : 0%| | 31.1MB / 22.2GB, 9.73MB/s New Data Upload : 5%|▌ | 31.1MB / 604MB, 9.73MB/s  ...nts/optim_045000_rank5.pt: 1%|▏ | 33.5MB / 2.49GB  Processing Files (0 / 1) : 0%| | 33.5MB / 22.2GB, 9.84MB/s New Data Upload : 6%|▌ | 33.5MB / 604MB, 9.84MB/s  ...nts/optim_045000_rank5.pt: 2%|▏ | 38.2MB / 2.49GB  Processing Files (0 / 1) : 0%| | 38.2MB / 22.2GB, 10.6MB/s New Data Upload : 6%|▋ | 38.2MB / 604MB, 10.6MB/s  ...nts/optim_045000_rank5.pt: 2%|▏ | 38.2MB / 2.49GB  ...nts/optim_045000_rank5.pt: 2%|▏ | 42.9MB / 2.49GB  Processing Files (0 / 1) : 0%| | 42.9MB / 22.2GB, 10.7MB/s New Data Upload : 7%|▋ | 42.9MB / 604MB, 10.7MB/s  ...nts/optim_045000_rank5.pt: 2%|▏ | 45.8MB / 2.49GB  Processing Files (0 / 1) : 0%| | 45.8MB / 22.2GB, 10.9MB/s New Data Upload : 8%|▊ | 45.8MB / 604MB, 10.9MB/s  ...nts/optim_045000_rank5.pt: 2%|▏ | 49.3MB / 2.49GB  Processing Files (0 / 1) : 0%| | 49.3MB / 22.2GB, 11.2MB/s New Data Upload : 8%|▊ | 49.3MB / 604MB, 11.2MB/s  ...nts/optim_045000_rank5.pt: 2%|▏ | 52.3MB / 2.49GB  Processing Files (0 / 1) : 0%| | 52.3MB / 22.2GB, 11.4MB/s New Data Upload : 9%|▊ | 52.3MB / 604MB, 11.4MB/s  ...nts/optim_045000_rank5.pt: 2%|▏ | 55.8MB / 2.49GB  Processing Files (0 / 1) : 0%| | 55.8MB / 22.2GB, 11.6MB/s New Data Upload : 9%|▉ | 55.8MB / 604MB, 11.6MB/s  ...nts/optim_045000_rank5.pt: 2%|▏ | 59.3MB / 2.49GB  Processing Files (0 / 1) : 0%| | 59.3MB / 22.2GB, 11.9MB/s New Data Upload : 10%|▉ | 59.3MB / 604MB, 11.9MB/s  ...nts/optim_045000_rank5.pt: 2%|▏ | 61.1MB / 2.49GB  Processing Files (0 / 1) : 0%| | 61.1MB / 22.2GB, 11.7MB/s New Data Upload : 10%|█ | 61.1MB / 604MB, 11.7MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 64.0MB / 2.49GB  Processing Files (0 / 1) : 0%| | 64.0MB / 22.2GB, 11.9MB/s New Data Upload : 11%|█ | 64.0MB / 604MB, 11.9MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 66.9MB / 2.49GB  Processing Files (0 / 1) : 0%| | 66.9MB / 22.2GB, 12.0MB/s New Data Upload : 11%|█ | 66.9MB / 604MB, 12.0MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 66.9MB / 2.49GB  ...nts/optim_045000_rank5.pt: 3%|▎ | 66.9MB / 2.49GB  ...nts/optim_045000_rank5.pt: 3%|▎ | 66.9MB / 2.49GB  ...nts/optim_045000_rank5.pt: 3%|▎ | 66.9MB / 2.49GB  ...nts/optim_045000_rank5.pt: 3%|▎ | 66.9MB / 2.49GB  ...nts/optim_045000_rank4.pt: 0%| | 4.11MB / 2.49GB  ...nts/optim_045000_rank5.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_045000_rank4.pt: 0%| | 4.11MB / 2.49GB  Processing Files (0 / 2) : 0%| | 71.2MB / 22.2GB, 10.5MB/s New Data Upload : 12%|█▏ | 71.2MB / 604MB, 10.5MB/s  ...nts/optim_045000_rank0.pt: 0%| | 587kB / 2.49GB  ...nts/optim_045000_rank1.pt: 0%| | 26.9kB / 2.49GB  ...nts/optim_045000_rank3.pt: 0%| | 26.9kB / 2.49GB  ...nts/optim_045000_rank6.pt: 0%| | 26.9kB / 2.49GB  ...nts/optim_045000_rank5.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_045000_rank4.pt: 0%| | 4.14MB / 2.49GB  ...nts/optim_045000_rank0.pt: 0%| | 587kB / 2.49GB  ...nts/optim_045000_rank1.pt: 0%| | 26.9kB / 2.49GB  ...nts/optim_045000_rank3.pt: 0%| | 26.9kB / 2.49GB  ...nts/optim_045000_rank6.pt: 0%| | 26.9kB / 2.49GB  Processing Files (0 / 6) : 0%| | 71.9MB / 22.2GB, 10.3MB/s New Data Upload : 11%|█ | 71.8MB / 671MB, 10.3MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_045000_rank4.pt: 0%| | 6.49MB / 2.49GB  ...nts/optim_045000_rank0.pt: 0%| | 587kB / 2.49GB  ...nts/optim_045000_rank1.pt: 0%| | 26.9kB / 2.49GB  ...nts/optim_045000_rank3.pt: 0%| | 26.9kB / 2.49GB  ...nts/optim_045000_rank6.pt: 0%| | 26.9kB / 2.49GB  Processing Files (0 / 6) : 0%| | 74.3MB / 22.2GB, 10.3MB/s New Data Upload : 10%|█ | 74.1MB / 738MB, 10.3MB/s  ...nts/optim_045000_rank7.pt: 0%| | 668kB / 2.49GB  ...nts/optim_045000_rank5.pt: 3%|▎ | 67.2MB / 2.49GB  ...nts/optim_045000_rank4.pt: 0%| | 8.33MB / 2.49GB  ...nts/optim_045000_rank0.pt: 0%| | 1.76MB / 2.49GB  ...nts/optim_045000_rank1.pt: 0%| | 107kB / 2.49GB  ...nts/optim_045000_rank3.pt: 0%| | 178kB / 2.49GB  ...nts/optim_045000_rank6.pt: 0%| | 107kB / 2.49GB  ...nts/optim_045000_rank7.pt: 0%| | 668kB / 2.49GB  Processing Files (0 / 7) : 0%| | 78.4MB / 22.2GB, 10.6MB/s New Data Upload : 11%|█ | 77.7MB / 738MB, 10.5MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 67.4MB / 2.49GB  ...nts/optim_045000_rank4.pt: 0%| | 8.54MB / 2.49GB  ...nts/optim_045000_rank0.pt: 0%| | 6.46MB / 2.49GB  ...nts/optim_045000_rank1.pt: 0%| | 323kB / 2.49GB  ...nts/optim_045000_rank3.pt: 0%| | 393kB / 2.49GB  ...nts/optim_045000_rank6.pt: 0%| | 323kB / 2.49GB  ...nts/optim_045000_rank7.pt: 0%| | 883kB / 2.49GB  Processing Files (0 / 7) : 0%| | 84.3MB / 22.2GB, 11.1MB/s New Data Upload : 11%|█ | 82.4MB / 738MB, 10.8MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 67.7MB / 2.49GB  ...nts/optim_045000_rank4.pt: 0%| | 10.6MB / 2.49GB  ...nts/optim_045000_rank0.pt: 0%| | 12.3MB / 2.49GB  ...nts/optim_045000_rank1.pt: 0%| | 618kB / 2.49GB  ...nts/optim_045000_rank3.pt: 0%| | 760kB / 2.49GB  ...nts/optim_045000_rank6.pt: 0%| | 618kB / 2.49GB  ...nts/optim_045000_rank7.pt: 0%| | 1.74MB / 2.49GB  Processing Files (0 / 7) : 0%| | 94.4MB / 22.2GB, 12.1MB/s New Data Upload : 11%|█▏ | 90.6MB / 805MB, 11.6MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 68.3MB / 2.49GB  ...nts/optim_045000_rank4.pt: 0%| | 11.2MB / 2.49GB  ...nts/optim_045000_rank0.pt: 1%| | 21.2MB / 2.49GB  ...nts/optim_045000_rank1.pt: 0%| | 1.18MB / 2.49GB  ...nts/optim_045000_rank3.pt: 0%| | 1.77MB / 2.49GB  ...nts/optim_045000_rank6.pt: 0%| | 1.77MB / 2.49GB  ...nts/optim_045000_rank7.pt: 0%| | 5.68MB / 2.49GB  Processing Files (0 / 7) : 0%| | 111MB / 22.2GB, 13.9MB/s New Data Upload : 13%|█▎ | 104MB / 805MB, 12.9MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 68.8MB / 2.49GB  ...nts/optim_045000_rank4.pt: 1%| | 14.0MB / 2.49GB  ...nts/optim_045000_rank0.pt: 1%| | 29.4MB / 2.49GB  ...nts/optim_045000_rank1.pt: 0%| | 1.67MB / 2.49GB  ...nts/optim_045000_rank3.pt: 0%| | 2.54MB / 2.49GB  ...nts/optim_045000_rank6.pt: 0%| | 2.25MB / 2.49GB  ...nts/optim_045000_rank7.pt: 0%| | 8.40MB / 2.49GB  Processing Files (0 / 7) : 1%| | 127MB / 22.2GB, 15.5MB/s New Data Upload : 14%|█▍ | 116MB / 805MB, 14.2MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 69.0MB / 2.49GB  ...nts/optim_045000_rank4.pt: 1%| | 14.2MB / 2.49GB  ...nts/optim_045000_rank0.pt: 1%|▏ | 32.3MB / 2.49GB  ...nts/optim_045000_rank1.pt: 0%| | 1.92MB / 2.49GB  ...nts/optim_045000_rank3.pt: 0%| | 3.02MB / 2.49GB  ...nts/optim_045000_rank6.pt: 0%| | 3.64MB / 2.49GB  ...nts/optim_045000_rank7.pt: 0%| | 10.3MB / 2.49GB  Processing Files (0 / 7) : 1%| | 134MB / 22.2GB, 16.0MB/s New Data Upload : 14%|█▍ | 122MB / 872MB, 14.6MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 69.4MB / 2.49GB  ...nts/optim_045000_rank4.pt: 1%| | 14.7MB / 2.49GB  ...nts/optim_045000_rank0.pt: 2%|▏ | 38.3MB / 2.49GB  ...nts/optim_045000_rank1.pt: 0%| | 2.94MB / 2.49GB  ...nts/optim_045000_rank3.pt: 0%| | 3.81MB / 2.49GB  ...nts/optim_045000_rank6.pt: 0%| | 8.69MB / 2.49GB  ...nts/optim_045000_rank7.pt: 1%| | 12.5MB / 2.49GB  Processing Files (0 / 7) : 1%| | 150MB / 22.2GB, 17.5MB/s New Data Upload : 16%|█▌ | 135MB / 872MB, 15.7MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 69.8MB / 2.49GB  ...nts/optim_045000_rank4.pt: 1%| | 16.8MB / 2.49GB  ...nts/optim_045000_rank0.pt: 2%|▏ | 44.8MB / 2.49GB  ...nts/optim_045000_rank1.pt: 0%| | 3.27MB / 2.49GB  ...nts/optim_045000_rank3.pt: 0%| | 4.19MB / 2.49GB  ...nts/optim_045000_rank6.pt: 0%| | 10.7MB / 2.49GB  ...nts/optim_045000_rank7.pt: 1%| | 12.8MB / 2.49GB  Processing Files (0 / 7) : 1%| | 162MB / 22.2GB, 18.4MB/s New Data Upload : 17%|█▋ | 145MB / 872MB, 16.5MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 70.4MB / 2.49GB  ...nts/optim_045000_rank4.pt: 1%| | 17.4MB / 2.49GB  ...nts/optim_045000_rank0.pt: 2%|▏ | 53.8MB / 2.49GB  ...nts/optim_045000_rank1.pt: 0%| | 5.64MB / 2.49GB  ...nts/optim_045000_rank3.pt: 0%| | 5.22MB / 2.49GB  ...nts/optim_045000_rank6.pt: 1%| | 17.7MB / 2.49GB  ...nts/optim_045000_rank7.pt: 1%| | 15.1MB / 2.49GB  Processing Files (0 / 7) : 1%| | 185MB / 22.2GB, 20.6MB/s New Data Upload : 19%|█▉ | 164MB / 872MB, 18.2MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 71.0MB / 2.49GB  ...nts/optim_045000_rank4.pt: 1%| | 18.0MB / 2.49GB  ...nts/optim_045000_rank0.pt: 3%|▎ | 63.3MB / 2.49GB  ...nts/optim_045000_rank1.pt: 0%| | 9.18MB / 2.49GB  ...nts/optim_045000_rank3.pt: 0%| | 6.69MB / 2.49GB  ...nts/optim_045000_rank6.pt: 1%| | 20.6MB / 2.49GB  ...nts/optim_045000_rank7.pt: 1%| | 17.5MB / 2.49GB  Processing Files (0 / 7) : 1%| | 206MB / 22.2GB, 22.4MB/s New Data Upload : 19%|█▉ | 181MB / 939MB, 19.7MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 71.5MB / 2.49GB  ...nts/optim_045000_rank4.pt: 1%| | 20.2MB / 2.49GB  ...nts/optim_045000_rank0.pt: 3%|▎ | 67.7MB / 2.49GB  ...nts/optim_045000_rank1.pt: 0%| | 12.0MB / 2.49GB  ...nts/optim_045000_rank3.pt: 0%| | 8.18MB / 2.49GB  ...nts/optim_045000_rank6.pt: 1%| | 29.1MB / 2.49GB  ...nts/optim_045000_rank7.pt: 1%| | 19.7MB / 2.49GB  Processing Files (0 / 7) : 1%| | 228MB / 22.2GB, 24.3MB/s New Data Upload : 21%|██▏ | 200MB / 939MB, 21.3MB/s  ...eckpoints/model_045000.pt: 0%| | 3.72MB / 2.35GB  ...nts/optim_045000_rank5.pt: 3%|▎ | 72.0MB / 2.49GB  ...nts/optim_045000_rank4.pt: 1%| | 20.7MB / 2.49GB  ...nts/optim_045000_rank0.pt: 3%|▎ | 68.2MB / 2.49GB  ...nts/optim_045000_rank1.pt: 1%| | 17.2MB / 2.49GB  ...nts/optim_045000_rank3.pt: 1%| | 12.9MB / 2.49GB  ...nts/optim_045000_rank6.pt: 1%|▏ | 34.3MB / 2.49GB  ...nts/optim_045000_rank7.pt: 1%| | 21.9MB / 2.49GB  ...eckpoints/model_045000.pt: 0%| | 3.72MB / 2.35GB  Processing Files (0 / 8) : 1%| | 251MB / 22.2GB, 26.1MB/s New Data Upload : 20%|██ | 216MB / 1.07GB, 22.5MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 72.0MB / 2.49GB  ...nts/optim_045000_rank4.pt: 1%| | 22.5MB / 2.49GB  ...nts/optim_045000_rank0.pt: 3%|▎ | 68.2MB / 2.49GB  ...nts/optim_045000_rank1.pt: 1%| | 17.2MB / 2.49GB  ...nts/optim_045000_rank3.pt: 1%| | 12.9MB / 2.49GB  ...nts/optim_045000_rank6.pt: 1%|▏ | 34.3MB / 2.49GB  ...nts/optim_045000_rank7.pt: 1%| | 21.9MB / 2.49GB  ...eckpoints/model_045000.pt: 0%| | 3.72MB / 2.35GB  Processing Files (0 / 8) : 1%| | 253MB / 22.2GB, 25.8MB/s New Data Upload : 20%|██ | 217MB / 1.07GB, 22.2MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 72.2MB / 2.49GB  ...nts/optim_045000_rank4.pt: 1%| | 22.7MB / 2.49GB  ...nts/optim_045000_rank0.pt: 3%|▎ | 68.5MB / 2.49GB  ...nts/optim_045000_rank1.pt: 1%| | 19.2MB / 2.49GB  ...nts/optim_045000_rank3.pt: 1%| | 14.9MB / 2.49GB  ...nts/optim_045000_rank6.pt: 2%|▏ | 39.1MB / 2.49GB  ...nts/optim_045000_rank7.pt: 1%| | 22.2MB / 2.49GB  ...eckpoints/model_045000.pt: 0%| | 4.31MB / 2.35GB  Processing Files (0 / 8) : 1%| | 263MB / 22.2GB, 26.3MB/s New Data Upload : 21%|██ | 226MB / 1.07GB, 22.6MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 74.4MB / 2.49GB  ...nts/optim_045000_rank4.pt: 1%| | 28.7MB / 2.49GB  ...nts/optim_045000_rank0.pt: 3%|▎ | 70.7MB / 2.49GB  ...nts/optim_045000_rank1.pt: 1%| | 23.1MB / 2.49GB  ...nts/optim_045000_rank3.pt: 1%| | 19.0MB / 2.49GB  ...nts/optim_045000_rank6.pt: 2%|▏ | 44.7MB / 2.49GB  ...nts/optim_045000_rank7.pt: 1%| | 26.5MB / 2.49GB  ...eckpoints/model_045000.pt: 0%| | 7.26MB / 2.35GB  Processing Files (0 / 8) : 1%|▏ | 294MB / 22.2GB, 28.9MB/s New Data Upload : 23%|██▎ | 244MB / 1.07GB, 23.9MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 75.7MB / 2.49GB  ...nts/optim_045000_rank4.pt: 1%|▏ | 35.6MB / 2.49GB  ...nts/optim_045000_rank0.pt: 3%|▎ | 72.1MB / 2.49GB  ...nts/optim_045000_rank1.pt: 1%| | 26.2MB / 2.49GB  ...nts/optim_045000_rank3.pt: 1%| | 20.3MB / 2.49GB  ...nts/optim_045000_rank6.pt: 2%|▏ | 47.2MB / 2.49GB  ...nts/optim_045000_rank7.pt: 1%| | 28.2MB / 2.49GB  ...eckpoints/model_045000.pt: 0%| | 11.4MB / 2.35GB  Processing Files (0 / 8) : 1%|▏ | 317MB / 22.2GB, 31.0MB/s New Data Upload : 24%|██▍ | 258MB / 1.07GB, 25.3MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 78.5MB / 2.49GB  ...nts/optim_045000_rank4.pt: 2%|▏ | 42.1MB / 2.49GB  ...nts/optim_045000_rank0.pt: 3%|▎ | 74.8MB / 2.49GB  ...nts/optim_045000_rank1.pt: 1%| | 30.7MB / 2.49GB  ...nts/optim_045000_rank3.pt: 1%| | 25.0MB / 2.49GB  ...nts/optim_045000_rank6.pt: 2%|▏ | 53.3MB / 2.49GB  ...nts/optim_045000_rank7.pt: 1%|▏ | 33.1MB / 2.49GB  ...eckpoints/model_045000.pt: 1%| | 14.4MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 352MB / 22.2GB, 34.5MB/s New Data Upload : 26%|██▌ | 276MB / 1.07GB, 27.0MB/s  ...nts/optim_045000_rank5.pt: 3%|▎ | 84.8MB / 2.49GB  ...nts/optim_045000_rank4.pt: 2%|▏ | 53.0MB / 2.49GB  ...nts/optim_045000_rank0.pt: 3%|▎ | 81.2MB / 2.49GB  ...nts/optim_045000_rank1.pt: 2%|▏ | 37.7MB / 2.49GB  ...nts/optim_045000_rank3.pt: 1%| | 30.8MB / 2.49GB  ...nts/optim_045000_rank6.pt: 2%|▏ | 60.9MB / 2.49GB  ...nts/optim_045000_rank7.pt: 2%|▏ | 39.4MB / 2.49GB  ...eckpoints/model_045000.pt: 1%| | 19.7MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 407MB / 22.2GB, 39.9MB/s New Data Upload : 28%|██▊ | 295MB / 1.07GB, 29.0MB/s  ...nts/optim_045000_rank5.pt: 4%|▎ | 93.0MB / 2.49GB  ...nts/optim_045000_rank4.pt: 3%|▎ | 63.3MB / 2.49GB  ...nts/optim_045000_rank0.pt: 4%|▎ | 89.4MB / 2.49GB  ...nts/optim_045000_rank1.pt: 2%|▏ | 46.0MB / 2.49GB  ...nts/optim_045000_rank3.pt: 2%|▏ | 39.8MB / 2.49GB  ...nts/optim_045000_rank6.pt: 3%|▎ | 68.6MB / 2.49GB  ...nts/optim_045000_rank7.pt: 2%|▏ | 47.4MB / 2.49GB  ...eckpoints/model_045000.pt: 1%| | 20.9MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 468MB / 22.2GB, 45.9MB/s New Data Upload : 29%|██▉ | 310MB / 1.07GB, 30.4MB/s  ...nts/optim_045000_rank5.pt: 4%|▍ | 99.5MB / 2.49GB  ...nts/optim_045000_rank4.pt: 3%|▎ | 75.0MB / 2.49GB  ...nts/optim_045000_rank0.pt: 4%|▍ | 95.8MB / 2.49GB  ...nts/optim_045000_rank1.pt: 2%|▏ | 53.2MB / 2.49GB  ...nts/optim_045000_rank3.pt: 2%|▏ | 47.5MB / 2.49GB  ...nts/optim_045000_rank6.pt: 3%|▎ | 75.8MB / 2.49GB  ...nts/optim_045000_rank7.pt: 2%|▏ | 55.1MB / 2.49GB  ...eckpoints/model_045000.pt: 1%| | 27.4MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 529MB / 22.2GB, 51.8MB/s New Data Upload : 31%|███ | 334MB / 1.07GB, 32.7MB/s  ...nts/optim_045000_rank5.pt: 4%|▍ | 109MB / 2.49GB  ...nts/optim_045000_rank4.pt: 4%|▎ | 88.7MB / 2.49GB  ...nts/optim_045000_rank0.pt: 4%|▍ | 106MB / 2.49GB  ...nts/optim_045000_rank1.pt: 3%|▎ | 64.4MB / 2.49GB  ...nts/optim_045000_rank3.pt: 2%|▏ | 58.4MB / 2.49GB  ...nts/optim_045000_rank6.pt: 4%|▎ | 87.0MB / 2.49GB  ...nts/optim_045000_rank7.pt: 3%|▎ | 65.1MB / 2.49GB  ...eckpoints/model_045000.pt: 1%| | 27.4MB / 2.35GB  Processing Files (0 / 8) : 3%|▎ | 606MB / 22.2GB, 59.4MB/s New Data Upload : 31%|███ | 354MB / 1.14GB, 34.6MB/s  ...nts/optim_045000_rank5.pt: 5%|▍ | 119MB / 2.49GB  ...nts/optim_045000_rank4.pt: 4%|▍ | 97.5MB / 2.49GB  ...nts/optim_045000_rank0.pt: 5%|▍ | 116MB / 2.49GB  ...nts/optim_045000_rank1.pt: 3%|▎ | 73.8MB / 2.49GB  ...nts/optim_045000_rank3.pt: 3%|▎ | 67.7MB / 2.49GB  ...nts/optim_045000_rank6.pt: 4%|▍ | 98.1MB / 2.49GB  ...nts/optim_045000_rank7.pt: 3%|▎ | 74.7MB / 2.49GB  ...eckpoints/model_045000.pt: 1%|▏ | 34.5MB / 2.35GB  Processing Files (0 / 8) : 3%|▎ | 681MB / 22.2GB, 66.6MB/s New Data Upload : 29%|██▉ | 375MB / 1.27GB, 36.6MB/s  ...nts/optim_045000_rank5.pt: 5%|▌ | 129MB / 2.49GB  ...nts/optim_045000_rank4.pt: 4%|▍ | 108MB / 2.49GB  ...nts/optim_045000_rank0.pt: 5%|▌ | 126MB / 2.49GB  ...nts/optim_045000_rank1.pt: 3%|▎ | 85.2MB / 2.49GB  ...nts/optim_045000_rank3.pt: 3%|▎ | 78.6MB / 2.49GB  ...nts/optim_045000_rank6.pt: 4%|▍ | 110MB / 2.49GB  ...nts/optim_045000_rank7.pt: 3%|▎ | 86.4MB / 2.49GB  ...eckpoints/model_045000.pt: 1%|▏ | 34.5MB / 2.35GB  Processing Files (0 / 8) : 3%|▎ | 756MB / 22.2GB, 73.7MB/s New Data Upload : 29%|██▉ | 396MB / 1.34GB, 38.4MB/s  ...nts/optim_045000_rank5.pt: 6%|▌ | 141MB / 2.49GB  ...nts/optim_045000_rank4.pt: 5%|▍ | 116MB / 2.49GB  ...nts/optim_045000_rank0.pt: 6%|▌ | 139MB / 2.49GB  ...nts/optim_045000_rank1.pt: 4%|▍ | 97.9MB / 2.49GB  ...nts/optim_045000_rank3.pt: 4%|▎ | 92.8MB / 2.49GB  ...nts/optim_045000_rank6.pt: 5%|▍ | 122MB / 2.49GB  ...nts/optim_045000_rank7.pt: 4%|▍ | 99.1MB / 2.49GB  ...eckpoints/model_045000.pt: 2%|▏ | 39.2MB / 2.35GB  Processing Files (0 / 8) : 4%|▍ | 847MB / 22.2GB, 82.3MB/s New Data Upload : 31%|███ | 416MB / 1.34GB, 39.9MB/s  ...nts/optim_045000_rank5.pt: 6%|▋ | 158MB / 2.49GB  ...nts/optim_045000_rank4.pt: 5%|▌ | 135MB / 2.49GB  ...nts/optim_045000_rank0.pt: 6%|▌ | 155MB / 2.49GB  ...nts/optim_045000_rank1.pt: 5%|▍ | 116MB / 2.49GB  ...nts/optim_045000_rank3.pt: 4%|▍ | 111MB / 2.49GB  ...nts/optim_045000_rank6.pt: 6%|▌ | 140MB / 2.49GB  ...nts/optim_045000_rank7.pt: 5%|▍ | 116MB / 2.49GB  ...eckpoints/model_045000.pt: 2%|▏ | 39.2MB / 2.35GB  Processing Files (0 / 8) : 4%|▍ | 970MB / 22.2GB, 93.8MB/s New Data Upload : 31%|███▏ | 442MB / 1.41GB, 42.1MB/s  ...nts/optim_045000_rank5.pt: 7%|▋ | 174MB / 2.49GB  ...nts/optim_045000_rank4.pt: 6%|▌ | 149MB / 2.49GB  ...nts/optim_045000_rank0.pt: 7%|▋ | 172MB / 2.49GB  ...nts/optim_045000_rank1.pt: 5%|▌ | 134MB / 2.49GB  ...nts/optim_045000_rank3.pt: 5%|▌ | 127MB / 2.49GB  ...nts/optim_045000_rank6.pt: 6%|▋ | 158MB / 2.49GB  ...nts/optim_045000_rank7.pt: 5%|▌ | 133MB / 2.49GB  ...eckpoints/model_045000.pt: 2%|▏ | 43.9MB / 2.35GB  Processing Files (0 / 8) : 5%|▍ | 1.09GB / 22.2GB, 105MB/s New Data Upload : 33%|███▎ | 469MB / 1.41GB, 44.5MB/s  ...nts/optim_045000_rank5.pt: 8%|▊ | 189MB / 2.49GB  ...nts/optim_045000_rank4.pt: 7%|▋ | 164MB / 2.49GB  ...nts/optim_045000_rank0.pt: 8%|▊ | 187MB / 2.49GB  ...nts/optim_045000_rank1.pt: 6%|▌ | 150MB / 2.49GB  ...nts/optim_045000_rank3.pt: 6%|▌ | 143MB / 2.49GB  ...nts/optim_045000_rank6.pt: 7%|▋ | 175MB / 2.49GB  ...nts/optim_045000_rank7.pt: 6%|▌ | 147MB / 2.49GB  ...eckpoints/model_045000.pt: 2%|▏ | 46.3MB / 2.35GB  Processing Files (0 / 8) : 5%|▌ | 1.20GB / 22.2GB, 116MB/s New Data Upload : 34%|███▎ | 496MB / 1.48GB, 46.8MB/s  ...nts/optim_045000_rank5.pt: 8%|▊ | 209MB / 2.49GB  ...nts/optim_045000_rank4.pt: 7%|▋ | 184MB / 2.49GB  ...nts/optim_045000_rank0.pt: 8%|▊ | 207MB / 2.49GB  ...nts/optim_045000_rank1.pt: 7%|▋ | 170MB / 2.49GB  ...nts/optim_045000_rank3.pt: 7%|▋ | 162MB / 2.49GB  ...nts/optim_045000_rank6.pt: 8%|▊ | 198MB / 2.49GB  ...nts/optim_045000_rank7.pt: 7%|▋ | 167MB / 2.49GB  ...eckpoints/model_045000.pt: 2%|▏ | 49.2MB / 2.35GB  Processing Files (0 / 8) : 6%|▌ | 1.35GB / 22.2GB, 130MB/s New Data Upload : 36%|███▌ | 525MB / 1.48GB, 49.3MB/s  ...nts/optim_045000_rank5.pt: 9%|▉ | 222MB / 2.49GB  ...nts/optim_045000_rank4.pt: 8%|▊ | 197MB / 2.49GB  ...nts/optim_045000_rank0.pt: 9%|▉ | 221MB / 2.49GB  ...nts/optim_045000_rank1.pt: 7%|▋ | 185MB / 2.49GB  ...nts/optim_045000_rank3.pt: 7%|▋ | 178MB / 2.49GB  ...nts/optim_045000_rank6.pt: 9%|▊ | 213MB / 2.49GB  ...nts/optim_045000_rank7.pt: 7%|▋ | 182MB / 2.49GB  ...eckpoints/model_045000.pt: 2%|▏ | 51.6MB / 2.35GB  Processing Files (0 / 8) : 7%|▋ | 1.45GB / 22.2GB, 140MB/s New Data Upload : 37%|███▋ | 550MB / 1.48GB, 51.5MB/s  ...nts/optim_045000_rank5.pt: 10%|▉ | 237MB / 2.49GB  ...nts/optim_045000_rank4.pt: 9%|▊ | 213MB / 2.49GB  ...nts/optim_045000_rank0.pt: 9%|▉ | 236MB / 2.49GB  ...nts/optim_045000_rank1.pt: 8%|▊ | 201MB / 2.49GB  ...nts/optim_045000_rank3.pt: 8%|▊ | 192MB / 2.49GB  ...nts/optim_045000_rank6.pt: 9%|▉ | 228MB / 2.49GB  ...nts/optim_045000_rank7.pt: 8%|▊ | 196MB / 2.49GB  ...eckpoints/model_045000.pt: 2%|▏ | 51.6MB / 2.35GB  Processing Files (0 / 8) : 7%|▋ | 1.55GB / 22.2GB, 150MB/s New Data Upload : 37%|███▋ | 571MB / 1.54GB, 53.1MB/s  ...nts/optim_045000_rank5.pt: 10%|▉ | 246MB / 2.49GB  ...nts/optim_045000_rank4.pt: 9%|▉ | 224MB / 2.49GB  ...nts/optim_045000_rank0.pt: 10%|▉ | 245MB / 2.49GB  ...nts/optim_045000_rank1.pt: 8%|▊ | 211MB / 2.49GB  ...nts/optim_045000_rank3.pt: 8%|▊ | 202MB / 2.49GB  ...nts/optim_045000_rank6.pt: 10%|▉ | 239MB / 2.49GB  ...nts/optim_045000_rank7.pt: 8%|▊ | 206MB / 2.49GB  ...eckpoints/model_045000.pt: 2%|▏ | 54.5MB / 2.35GB  Processing Files (0 / 8) : 7%|▋ | 1.63GB / 22.2GB, 156MB/s New Data Upload : 37%|███▋ | 588MB / 1.61GB, 54.6MB/s  ...nts/optim_045000_rank5.pt: 11%|█ | 266MB / 2.49GB  ...nts/optim_045000_rank4.pt: 10%|▉ | 245MB / 2.49GB  ...nts/optim_045000_rank0.pt: 11%|█ | 265MB / 2.49GB  ...nts/optim_045000_rank1.pt: 9%|▉ | 233MB / 2.49GB  ...nts/optim_045000_rank3.pt: 9%|▉ | 223MB / 2.49GB  ...nts/optim_045000_rank6.pt: 11%|█ | 262MB / 2.49GB  ...nts/optim_045000_rank7.pt: 9%|▉ | 226MB / 2.49GB  ...eckpoints/model_045000.pt: 2%|▏ | 56.9MB / 2.35GB  Processing Files (0 / 8) : 8%|▊ | 1.78GB / 22.2GB, 171MB/s New Data Upload : 39%|███▊ | 620MB / 1.61GB, 57.5MB/s  ...nts/optim_045000_rank5.pt: 11%|█ | 276MB / 2.49GB  ...nts/optim_045000_rank4.pt: 10%|█ | 256MB / 2.49GB  ...nts/optim_045000_rank0.pt: 11%|█ | 275MB / 2.49GB  ...nts/optim_045000_rank1.pt: 10%|▉ | 244MB / 2.49GB  ...nts/optim_045000_rank3.pt: 9%|▉ | 235MB / 2.49GB  ...nts/optim_045000_rank6.pt: 11%|█ | 273MB / 2.49GB  ...nts/optim_045000_rank7.pt: 9%|▉ | 235MB / 2.49GB  ...eckpoints/model_045000.pt: 3%|▎ | 59.8MB / 2.35GB  Processing Files (0 / 8) : 8%|▊ | 1.85GB / 22.2GB, 178MB/s New Data Upload : 40%|███▉ | 640MB / 1.61GB, 59.0MB/s  ...nts/optim_045000_rank5.pt: 12%|█▏ | 292MB / 2.49GB  ...nts/optim_045000_rank4.pt: 11%|█ | 270MB / 2.49GB  ...nts/optim_045000_rank0.pt: 12%|█▏ | 291MB / 2.49GB  ...nts/optim_045000_rank1.pt: 11%|█ | 261MB / 2.49GB  ...nts/optim_045000_rank3.pt: 10%|█ | 253MB / 2.49GB  ...nts/optim_045000_rank6.pt: 12%|█▏ | 291MB / 2.49GB  ...nts/optim_045000_rank7.pt: 10%|█ | 253MB / 2.49GB  ...eckpoints/model_045000.pt: 3%|▎ | 65.7MB / 2.35GB  Processing Files (0 / 8) : 9%|▉ | 1.98GB / 22.2GB, 190MB/s New Data Upload : 42%|████▏ | 669MB / 1.61GB, 61.9MB/s  ...nts/optim_045000_rank5.pt: 12%|█▏ | 300MB / 2.49GB  ...nts/optim_045000_rank4.pt: 11%|█ | 279MB / 2.49GB  ...nts/optim_045000_rank0.pt: 12%|█▏ | 299MB / 2.49GB  ...nts/optim_045000_rank1.pt: 11%|█ | 268MB / 2.49GB  ...nts/optim_045000_rank3.pt: 10%|█ | 261MB / 2.49GB  ...nts/optim_045000_rank6.pt: 12%|█▏ | 300MB / 2.49GB  ...nts/optim_045000_rank7.pt: 10%|█ | 261MB / 2.49GB  ...eckpoints/model_045000.pt: 3%|▎ | 70.4MB / 2.35GB  Processing Files (0 / 8) : 9%|▉ | 2.04GB / 22.2GB, 196MB/s New Data Upload : 41%|████ | 684MB / 1.68GB, 62.9MB/s  ...nts/optim_045000_rank5.pt: 13%|█▎ | 316MB / 2.49GB  ...nts/optim_045000_rank4.pt: 12%|█▏ | 293MB / 2.49GB  ...nts/optim_045000_rank0.pt: 13%|█▎ | 319MB / 2.49GB  ...nts/optim_045000_rank1.pt: 12%|█▏ | 289MB / 2.49GB  ...nts/optim_045000_rank3.pt: 11%|█▏ | 281MB / 2.49GB  ...nts/optim_045000_rank6.pt: 13%|█▎ | 319MB / 2.49GB  ...nts/optim_045000_rank7.pt: 11%|█ | 277MB / 2.49GB  ...eckpoints/model_045000.pt: 3%|▎ | 73.8MB / 2.35GB  Processing Files (0 / 8) : 10%|▉ | 2.17GB / 22.2GB, 208MB/s New Data Upload : 43%|████▎ | 714MB / 1.68GB, 65.5MB/s  ...nts/optim_045000_rank5.pt: 13%|█▎ | 326MB / 2.49GB  ...nts/optim_045000_rank4.pt: 12%|█▏ | 303MB / 2.49GB  ...nts/optim_045000_rank0.pt: 13%|█▎ | 328MB / 2.49GB  ...nts/optim_045000_rank1.pt: 12%|█▏ | 298MB / 2.49GB  ...nts/optim_045000_rank3.pt: 12%|█▏ | 292MB / 2.49GB  ...nts/optim_045000_rank6.pt: 13%|█▎ | 329MB / 2.49GB  ...nts/optim_045000_rank7.pt: 12%|█▏ | 286MB / 2.49GB  ...eckpoints/model_045000.pt: 3%|▎ | 81.4MB / 2.35GB  Processing Files (0 / 8) : 10%|█ | 2.24GB / 22.2GB, 215MB/s New Data Upload : 44%|████▎ | 733MB / 1.68GB, 67.0MB/s  ...nts/optim_045000_rank5.pt: 14%|█▎ | 339MB / 2.49GB  ...nts/optim_045000_rank4.pt: 13%|█▎ | 318MB / 2.49GB  ...nts/optim_045000_rank0.pt: 14%|█▎ | 341MB / 2.49GB  ...nts/optim_045000_rank1.pt: 13%|█▎ | 312MB / 2.49GB  ...nts/optim_045000_rank3.pt: 12%|█▏ | 305MB / 2.49GB  ...nts/optim_045000_rank6.pt: 14%|█▍ | 343MB / 2.49GB  ...nts/optim_045000_rank7.pt: 12%|█▏ | 299MB / 2.49GB  ...eckpoints/model_045000.pt: 4%|▎ | 87.8MB / 2.35GB  Processing Files (0 / 8) : 11%|█ | 2.34GB / 22.2GB, 225MB/s New Data Upload : 44%|████▎ | 759MB / 1.74GB, 69.3MB/s  ...nts/optim_045000_rank5.pt: 14%|█▍ | 350MB / 2.49GB  ...nts/optim_045000_rank4.pt: 13%|█▎ | 329MB / 2.49GB  ...nts/optim_045000_rank0.pt: 14%|█▍ | 355MB / 2.49GB  ...nts/optim_045000_rank1.pt: 13%|█▎ | 327MB / 2.49GB  ...nts/optim_045000_rank3.pt: 13%|█▎ | 320MB / 2.49GB  ...nts/optim_045000_rank6.pt: 14%|█▍ | 356MB / 2.49GB  ...nts/optim_045000_rank7.pt: 13%|█▎ | 311MB / 2.49GB  ...eckpoints/model_045000.pt: 4%|▍ | 96.0MB / 2.35GB  Processing Files (0 / 8) : 11%|█ | 2.44GB / 22.2GB, 234MB/s New Data Upload : 44%|████▎ | 788MB / 1.81GB, 71.8MB/s  ...nts/optim_045000_rank5.pt: 15%|█▍ | 365MB / 2.49GB  ...nts/optim_045000_rank4.pt: 14%|█▍ | 346MB / 2.49GB  ...nts/optim_045000_rank0.pt: 15%|█▍ | 371MB / 2.49GB  ...nts/optim_045000_rank1.pt: 14%|█▍ | 342MB / 2.49GB  ...nts/optim_045000_rank3.pt: 14%|█▎ | 336MB / 2.49GB  ...nts/optim_045000_rank6.pt: 15%|█▍ | 372MB / 2.49GB  ...nts/optim_045000_rank7.pt: 13%|█▎ | 326MB / 2.49GB  ...eckpoints/model_045000.pt: 4%|▍ | 100MB / 2.35GB  Processing Files (0 / 8) : 12%|█▏ | 2.56GB / 22.2GB, 245MB/s New Data Upload : 45%|████▍ | 812MB / 1.81GB, 73.8MB/s  ...nts/optim_045000_rank5.pt: 15%|█▌ | 373MB / 2.49GB  ...nts/optim_045000_rank4.pt: 14%|█▍ | 354MB / 2.49GB  ...nts/optim_045000_rank0.pt: 15%|█▌ | 381MB / 2.49GB  ...nts/optim_045000_rank1.pt: 14%|█▍ | 353MB / 2.49GB  ...nts/optim_045000_rank3.pt: 14%|█▍ | 347MB / 2.49GB  ...nts/optim_045000_rank6.pt: 15%|█▌ | 382MB / 2.49GB  ...nts/optim_045000_rank7.pt: 13%|█▎ | 334MB / 2.49GB  ...eckpoints/model_045000.pt: 4%|▍ | 105MB / 2.35GB  Processing Files (0 / 8) : 12%|█▏ | 2.63GB / 22.2GB, 252MB/s New Data Upload : 46%|████▌ | 832MB / 1.81GB, 75.5MB/s  ...nts/optim_045000_rank5.pt: 15%|█▌ | 377MB / 2.49GB  ...nts/optim_045000_rank4.pt: 14%|█▍ | 359MB / 2.49GB  ...nts/optim_045000_rank0.pt: 16%|█▌ | 387MB / 2.49GB  ...nts/optim_045000_rank1.pt: 14%|█▍ | 359MB / 2.49GB  ...nts/optim_045000_rank3.pt: 14%|█▍ | 353MB / 2.49GB  ...nts/optim_045000_rank6.pt: 16%|█▌ | 388MB / 2.49GB  ...nts/optim_045000_rank7.pt: 14%|█▎ | 339MB / 2.49GB  ...eckpoints/model_045000.pt: 5%|▍ | 111MB / 2.35GB  Processing Files (0 / 8) : 12%|█▏ | 2.67GB / 22.2GB, 256MB/s New Data Upload : 47%|████▋ | 845MB / 1.81GB, 76.6MB/s  ...nts/optim_045000_rank5.pt: 16%|█▌ | 388MB / 2.49GB  ...nts/optim_045000_rank4.pt: 15%|█▍ | 371MB / 2.49GB  ...nts/optim_045000_rank0.pt: 16%|█▌ | 400MB / 2.49GB  ...nts/optim_045000_rank1.pt: 15%|█▍ | 372MB / 2.49GB  ...nts/optim_045000_rank3.pt: 15%|█▍ | 366MB / 2.49GB  ...nts/optim_045000_rank6.pt: 16%|█▌ | 400MB / 2.49GB  ...nts/optim_045000_rank7.pt: 14%|█▍ | 350MB / 2.49GB  ...eckpoints/model_045000.pt: 5%|▍ | 117MB / 2.35GB  Processing Files (0 / 8) : 12%|█▏ | 2.76GB / 22.2GB, 264MB/s New Data Upload : 46%|████▋ | 871MB / 1.88GB, 78.8MB/s  ...nts/optim_045000_rank5.pt: 16%|█▌ | 400MB / 2.49GB  ...nts/optim_045000_rank4.pt: 15%|█▌ | 382MB / 2.49GB  ...nts/optim_045000_rank0.pt: 17%|█▋ | 413MB / 2.49GB  ...nts/optim_045000_rank1.pt: 16%|█▌ | 387MB / 2.49GB  ...nts/optim_045000_rank3.pt: 15%|█▌ | 379MB / 2.49GB  ...nts/optim_045000_rank6.pt: 17%|█▋ | 413MB / 2.49GB  ...nts/optim_045000_rank7.pt: 15%|█▍ | 363MB / 2.49GB  ...eckpoints/model_045000.pt: 5%|▌ | 127MB / 2.35GB  Processing Files (0 / 8) : 13%|█▎ | 2.86GB / 22.2GB, 274MB/s New Data Upload : 43%|████▎ | 898MB / 2.08GB, 81.5MB/s  ...nts/optim_045000_rank5.pt: 16%|█▋ | 407MB / 2.49GB  ...nts/optim_045000_rank4.pt: 16%|█▌ | 390MB / 2.49GB  ...nts/optim_045000_rank0.pt: 17%|█▋ | 421MB / 2.49GB  ...nts/optim_045000_rank1.pt: 16%|█▌ | 395MB / 2.49GB  ...nts/optim_045000_rank3.pt: 16%|█▌ | 389MB / 2.49GB  ...nts/optim_045000_rank6.pt: 17%|█▋ | 421MB / 2.49GB  ...nts/optim_045000_rank7.pt: 15%|█▍ | 371MB / 2.49GB  ...eckpoints/model_045000.pt: 6%|▌ | 132MB / 2.35GB  Processing Files (0 / 8) : 13%|█▎ | 2.93GB / 22.2GB, 280MB/s New Data Upload : 44%|████▍ | 916MB / 2.08GB, 83.3MB/s  ...nts/optim_045000_rank5.pt: 17%|█▋ | 416MB / 2.49GB  ...nts/optim_045000_rank4.pt: 16%|█▌ | 400MB / 2.49GB  ...nts/optim_045000_rank0.pt: 17%|█▋ | 432MB / 2.49GB  ...nts/optim_045000_rank1.pt: 16%|█▋ | 406MB / 2.49GB  ...nts/optim_045000_rank3.pt: 16%|█▌ | 399MB / 2.49GB  ...nts/optim_045000_rank6.pt: 17%|█▋ | 431MB / 2.49GB  ...nts/optim_045000_rank7.pt: 15%|█▌ | 380MB / 2.49GB  ...eckpoints/model_045000.pt: 6%|▌ | 135MB / 2.35GB  Processing Files (0 / 8) : 13%|█▎ | 3.00GB / 22.2GB, 288MB/s New Data Upload : 44%|████▎ | 935MB / 2.15GB, 85.1MB/s  ...nts/optim_045000_rank5.pt: 17%|█▋ | 428MB / 2.49GB  ...nts/optim_045000_rank4.pt: 17%|█▋ | 414MB / 2.49GB  ...nts/optim_045000_rank0.pt: 18%|█▊ | 446MB / 2.49GB  ...nts/optim_045000_rank1.pt: 17%|█▋ | 422MB / 2.49GB  ...nts/optim_045000_rank3.pt: 17%|█▋ | 415MB / 2.49GB  ...nts/optim_045000_rank6.pt: 18%|█▊ | 445MB / 2.49GB  ...nts/optim_045000_rank7.pt: 16%|█▌ | 392MB / 2.49GB  ...eckpoints/model_045000.pt: 6%|▌ | 140MB / 2.35GB  Processing Files (0 / 8) : 14%|█▍ | 3.10GB / 22.2GB, 297MB/s New Data Upload : 45%|████▍ | 959MB / 2.15GB, 87.5MB/s  ...nts/optim_045000_rank5.pt: 18%|█▊ | 439MB / 2.49GB  ...nts/optim_045000_rank4.pt: 17%|█▋ | 424MB / 2.49GB  ...nts/optim_045000_rank0.pt: 18%|█▊ | 458MB / 2.49GB  ...nts/optim_045000_rank1.pt: 17%|█▋ | 433MB / 2.49GB  ...nts/optim_045000_rank3.pt: 17%|█▋ | 426MB / 2.49GB  ...nts/optim_045000_rank6.pt: 18%|█▊ | 456MB / 2.49GB  ...nts/optim_045000_rank7.pt: 16%|█▋ | 406MB / 2.49GB  ...eckpoints/model_045000.pt: 6%|▌ | 143MB / 2.35GB  Processing Files (0 / 8) : 14%|█▍ | 3.18GB / 22.2GB, 306MB/s New Data Upload : 43%|████▎ | 980MB / 2.28GB, 89.5MB/s  ...nts/optim_045000_rank5.pt: 18%|█▊ | 453MB / 2.49GB  ...nts/optim_045000_rank4.pt: 18%|█▊ | 438MB / 2.49GB  ...nts/optim_045000_rank0.pt: 19%|█▉ | 472MB / 2.49GB  ...nts/optim_045000_rank1.pt: 18%|█▊ | 449MB / 2.49GB  ...nts/optim_045000_rank3.pt: 18%|█▊ | 442MB / 2.49GB  ...nts/optim_045000_rank6.pt: 19%|█▉ | 471MB / 2.49GB  ...nts/optim_045000_rank7.pt: 17%|█▋ | 421MB / 2.49GB  ...eckpoints/model_045000.pt: 6%|▌ | 143MB / 2.35GB  Processing Files (0 / 8) : 15%|█▍ | 3.29GB / 22.2GB, 315MB/s New Data Upload : 43%|████▎ | 1.00GB / 2.35GB, 91.1MB/s  ...nts/optim_045000_rank5.pt: 19%|█▉ | 467MB / 2.49GB  ...nts/optim_045000_rank4.pt: 18%|█▊ | 453MB / 2.49GB  ...nts/optim_045000_rank0.pt: 20%|█▉ | 487MB / 2.49GB  ...nts/optim_045000_rank1.pt: 19%|█▊ | 465MB / 2.49GB  ...nts/optim_045000_rank3.pt: 18%|█▊ | 458MB / 2.49GB  ...nts/optim_045000_rank6.pt: 20%|█▉ | 485MB / 2.49GB  ...nts/optim_045000_rank7.pt: 18%|█▊ | 435MB / 2.49GB  ...eckpoints/model_045000.pt: 6%|▌ | 144MB / 2.35GB  Processing Files (0 / 8) : 15%|█▌ | 3.39GB / 22.2GB, 326MB/s New Data Upload : 42%|████▏ | 1.02GB / 2.41GB, 93.3MB/s  ...nts/optim_045000_rank5.pt: 20%|█▉ | 494MB / 2.49GB  ...nts/optim_045000_rank4.pt: 19%|█▉ | 481MB / 2.49GB  ...nts/optim_045000_rank0.pt: 21%|██ | 515MB / 2.49GB  ...nts/optim_045000_rank1.pt: 20%|█▉ | 492MB / 2.49GB  ...nts/optim_045000_rank3.pt: 20%|█▉ | 486MB / 2.49GB  ...nts/optim_045000_rank6.pt: 21%|██ | 516MB / 2.49GB  ...nts/optim_045000_rank7.pt: 19%|█▊ | 465MB / 2.49GB  ...eckpoints/model_045000.pt: 6%|▌ | 144MB / 2.35GB  Processing Files (0 / 8) : 16%|█▌ | 3.59GB / 22.2GB, 345MB/s New Data Upload : 44%|████▍ | 1.06GB / 2.41GB, 97.0MB/s  ...nts/optim_045000_rank5.pt: 20%|██ | 508MB / 2.49GB  ...nts/optim_045000_rank4.pt: 20%|█▉ | 496MB / 2.49GB  ...nts/optim_045000_rank0.pt: 21%|██▏ | 528MB / 2.49GB  ...nts/optim_045000_rank1.pt: 20%|██ | 507MB / 2.49GB  ...nts/optim_045000_rank3.pt: 20%|██ | 501MB / 2.49GB  ...nts/optim_045000_rank6.pt: 21%|██▏ | 529MB / 2.49GB  ...nts/optim_045000_rank7.pt: 19%|█▉ | 480MB / 2.49GB  ...eckpoints/model_045000.pt: 6%|▌ | 144MB / 2.35GB  Processing Files (0 / 8) : 17%|█▋ | 3.69GB / 22.2GB, 354MB/s New Data Upload : 44%|████▎ | 1.08GB / 2.48GB, 98.7MB/s  ...nts/optim_045000_rank5.pt: 21%|██▏ | 534MB / 2.49GB  ...nts/optim_045000_rank4.pt: 21%|██ | 522MB / 2.49GB  ...nts/optim_045000_rank0.pt: 22%|██▏ | 557MB / 2.49GB  ...nts/optim_045000_rank1.pt: 22%|██▏ | 536MB / 2.49GB  ...nts/optim_045000_rank3.pt: 21%|██ | 527MB / 2.49GB  ...nts/optim_045000_rank6.pt: 22%|██▏ | 553MB / 2.49GB  ...nts/optim_045000_rank7.pt: 20%|██ | 506MB / 2.49GB  ...eckpoints/model_045000.pt: 6%|▌ | 147MB / 2.35GB  Processing Files (0 / 8) : 17%|█▋ | 3.88GB / 22.2GB, 372MB/s New Data Upload : 44%|████▍ | 1.12GB / 2.55GB, 102MB/s  ...nts/optim_045000_rank5.pt: 22%|██▏ | 549MB / 2.49GB  ...nts/optim_045000_rank4.pt: 22%|██▏ | 537MB / 2.49GB  ...nts/optim_045000_rank0.pt: 23%|██▎ | 572MB / 2.49GB  ...nts/optim_045000_rank1.pt: 22%|██▏ | 552MB / 2.49GB  ...nts/optim_045000_rank3.pt: 22%|██▏ | 545MB / 2.49GB  ...nts/optim_045000_rank6.pt: 23%|██▎ | 569MB / 2.49GB  ...nts/optim_045000_rank7.pt: 21%|██ | 523MB / 2.49GB  ...eckpoints/model_045000.pt: 6%|▌ | 147MB / 2.35GB  Processing Files (0 / 8) : 18%|█▊ | 3.99GB / 22.2GB, 382MB/s New Data Upload : 45%|████▌ | 1.15GB / 2.55GB, 104MB/s  ...nts/optim_045000_rank5.pt: 23%|██▎ | 573MB / 2.49GB  ...nts/optim_045000_rank4.pt: 23%|██▎ | 559MB / 2.49GB  ...nts/optim_045000_rank0.pt: 24%|██▍ | 595MB / 2.49GB  ...nts/optim_045000_rank1.pt: 23%|██▎ | 578MB / 2.49GB  ...nts/optim_045000_rank3.pt: 23%|██▎ | 567MB / 2.49GB  ...nts/optim_045000_rank6.pt: 24%|██▎ | 590MB / 2.49GB  ...nts/optim_045000_rank7.pt: 22%|██▏ | 546MB / 2.49GB  ...eckpoints/model_045000.pt: 6%|▋ | 148MB / 2.35GB  Processing Files (0 / 8) : 19%|█▊ | 4.16GB / 22.2GB, 397MB/s New Data Upload : 44%|████▍ | 1.18GB / 2.68GB, 106MB/s  ...nts/optim_045000_rank5.pt: 24%|██▍ | 597MB / 2.49GB  ...nts/optim_045000_rank4.pt: 24%|██▎ | 586MB / 2.49GB  ...nts/optim_045000_rank0.pt: 25%|██▍ | 621MB / 2.49GB  ...nts/optim_045000_rank1.pt: 24%|██▍ | 604MB / 2.49GB  ...nts/optim_045000_rank3.pt: 24%|██▍ | 592MB / 2.49GB  ...nts/optim_045000_rank6.pt: 25%|██▍ | 612MB / 2.49GB  ...nts/optim_045000_rank7.pt: 23%|██▎ | 566MB / 2.49GB  ...eckpoints/model_045000.pt: 6%|▋ | 151MB / 2.35GB  Processing Files (0 / 8) : 19%|█▉ | 4.33GB / 22.2GB, 412MB/s New Data Upload : 45%|████▌ | 1.21GB / 2.68GB, 108MB/s  ...nts/optim_045000_rank5.pt: 25%|██▍ | 620MB / 2.49GB  ...nts/optim_045000_rank4.pt: 25%|██▍ | 610MB / 2.49GB  ...nts/optim_045000_rank0.pt: 26%|██▌ | 644MB / 2.49GB  ...nts/optim_045000_rank1.pt: 25%|██▌ | 628MB / 2.49GB  ...nts/optim_045000_rank3.pt: 25%|██▍ | 616MB / 2.49GB  ...nts/optim_045000_rank6.pt: 25%|██▌ | 633MB / 2.49GB  ...nts/optim_045000_rank7.pt: 24%|██▎ | 586MB / 2.49GB  ...eckpoints/model_045000.pt: 7%|▋ | 155MB / 2.35GB  Processing Files (0 / 8) : 20%|██ | 4.49GB / 22.2GB, 427MB/s New Data Upload : 47%|████▋ | 1.25GB / 2.68GB, 111MB/s  ...nts/optim_045000_rank5.pt: 26%|██▌ | 635MB / 2.49GB  ...nts/optim_045000_rank4.pt: 25%|██▌ | 625MB / 2.49GB  ...nts/optim_045000_rank0.pt: 27%|██▋ | 659MB / 2.49GB  ...nts/optim_045000_rank1.pt: 26%|██▌ | 643MB / 2.49GB  ...nts/optim_045000_rank3.pt: 25%|██▌ | 631MB / 2.49GB  ...nts/optim_045000_rank6.pt: 26%|██▌ | 648MB / 2.49GB  ...nts/optim_045000_rank7.pt: 24%|██▍ | 601MB / 2.49GB  ...eckpoints/model_045000.pt: 7%|▋ | 160MB / 2.35GB  Processing Files (0 / 8) : 21%|██ | 4.60GB / 22.2GB, 436MB/s New Data Upload : 47%|████▋ | 1.27GB / 2.68GB, 112MB/s  ...nts/optim_045000_rank5.pt: 26%|██▋ | 655MB / 2.49GB  ...nts/optim_045000_rank4.pt: 26%|██▌ | 646MB / 2.49GB  ...nts/optim_045000_rank0.pt: 27%|██▋ | 680MB / 2.49GB  ...nts/optim_045000_rank1.pt: 27%|██▋ | 665MB / 2.49GB  ...nts/optim_045000_rank3.pt: 26%|██▋ | 653MB / 2.49GB  ...nts/optim_045000_rank6.pt: 27%|██▋ | 668MB / 2.49GB  ...nts/optim_045000_rank7.pt: 25%|██▍ | 620MB / 2.49GB  ...eckpoints/model_045000.pt: 7%|▋ | 164MB / 2.35GB  Processing Files (0 / 8) : 21%|██▏ | 4.75GB / 22.2GB, 450MB/s New Data Upload : 49%|████▉ | 1.31GB / 2.68GB, 114MB/s  ...nts/optim_045000_rank5.pt: 27%|██▋ | 674MB / 2.49GB  ...nts/optim_045000_rank4.pt: 27%|██▋ | 663MB / 2.49GB  ...nts/optim_045000_rank0.pt: 28%|██▊ | 699MB / 2.49GB  ...nts/optim_045000_rank1.pt: 28%|██▊ | 684MB / 2.49GB  ...nts/optim_045000_rank3.pt: 27%|██▋ | 672MB / 2.49GB  ...nts/optim_045000_rank6.pt: 28%|██▊ | 686MB / 2.49GB  ...nts/optim_045000_rank7.pt: 26%|██▌ | 637MB / 2.49GB  ...eckpoints/model_045000.pt: 7%|▋ | 168MB / 2.35GB  Processing Files (0 / 8) : 22%|██▏ | 4.88GB / 22.2GB, 461MB/s New Data Upload : 50%|████▉ | 1.34GB / 2.68GB, 115MB/s  ...nts/optim_045000_rank5.pt: 28%|██▊ | 707MB / 2.49GB  ...nts/optim_045000_rank4.pt: 28%|██▊ | 694MB / 2.49GB  ...nts/optim_045000_rank0.pt: 29%|██▉ | 730MB / 2.49GB  ...nts/optim_045000_rank1.pt: 29%|██▉ | 718MB / 2.49GB  ...nts/optim_045000_rank3.pt: 28%|██▊ | 707MB / 2.49GB  ...nts/optim_045000_rank6.pt: 29%|██▉ | 718MB / 2.49GB  ...nts/optim_045000_rank7.pt: 27%|██▋ | 669MB / 2.49GB  ...eckpoints/model_045000.pt: 7%|▋ | 174MB / 2.35GB  Processing Files (0 / 8) : 23%|██▎ | 5.12GB / 22.2GB, 482MB/s New Data Upload : 50%|█████ | 1.39GB / 2.75GB, 118MB/s  ...nts/optim_045000_rank5.pt: 29%|██▉ | 729MB / 2.49GB  ...nts/optim_045000_rank4.pt: 29%|██▊ | 715MB / 2.49GB  ...nts/optim_045000_rank0.pt: 30%|███ | 753MB / 2.49GB  ...nts/optim_045000_rank1.pt: 30%|██▉ | 741MB / 2.49GB  ...nts/optim_045000_rank3.pt: 29%|██▉ | 730MB / 2.49GB  ...nts/optim_045000_rank6.pt: 30%|██▉ | 739MB / 2.49GB  ...nts/optim_045000_rank7.pt: 28%|██▊ | 690MB / 2.49GB  ...eckpoints/model_045000.pt: 8%|▊ | 178MB / 2.35GB  Processing Files (0 / 8) : 24%|██▎ | 5.27GB / 22.2GB, 495MB/s New Data Upload : 50%|█████ | 1.42GB / 2.82GB, 119MB/s  ...nts/optim_045000_rank5.pt: 30%|███ | 750MB / 2.49GB  ...nts/optim_045000_rank4.pt: 29%|██▉ | 733MB / 2.49GB  ...nts/optim_045000_rank0.pt: 31%|███ | 773MB / 2.49GB  ...nts/optim_045000_rank1.pt: 31%|███ | 763MB / 2.49GB  ...nts/optim_045000_rank3.pt: 30%|███ | 753MB / 2.49GB  ...nts/optim_045000_rank6.pt: 31%|███ | 759MB / 2.49GB  ...nts/optim_045000_rank7.pt: 29%|██▊ | 711MB / 2.49GB  ...eckpoints/model_045000.pt: 8%|▊ | 183MB / 2.35GB  Processing Files (0 / 8) : 24%|██▍ | 5.42GB / 22.2GB, 507MB/s New Data Upload : 52%|█████▏ | 1.45GB / 2.82GB, 121MB/s  ...nts/optim_045000_rank5.pt: 31%|███ | 774MB / 2.49GB  ...nts/optim_045000_rank4.pt: 30%|███ | 752MB / 2.49GB  ...nts/optim_045000_rank0.pt: 32%|███▏ | 799MB / 2.49GB  ...nts/optim_045000_rank1.pt: 32%|███▏ | 788MB / 2.49GB  ...nts/optim_045000_rank3.pt: 31%|███▏ | 777MB / 2.49GB  ...nts/optim_045000_rank6.pt: 31%|███▏ | 783MB / 2.49GB  ...nts/optim_045000_rank7.pt: 30%|██▉ | 733MB / 2.49GB  ...eckpoints/model_045000.pt: 8%|▊ | 187MB / 2.35GB  Processing Files (0 / 8) : 25%|██▌ | 5.59GB / 22.2GB, 523MB/s New Data Upload : 51%|█████ | 1.49GB / 2.95GB, 125MB/s  ...nts/optim_045000_rank5.pt: 32%|███▏ | 796MB / 2.49GB  ...nts/optim_045000_rank4.pt: 31%|███▏ | 779MB / 2.49GB  ...nts/optim_045000_rank0.pt: 33%|███▎ | 821MB / 2.49GB  ...nts/optim_045000_rank1.pt: 33%|███▎ | 809MB / 2.49GB  ...nts/optim_045000_rank3.pt: 32%|███▏ | 799MB / 2.49GB  ...nts/optim_045000_rank6.pt: 32%|███▏ | 805MB / 2.49GB  ...nts/optim_045000_rank7.pt: 30%|███ | 754MB / 2.49GB  ...eckpoints/model_045000.pt: 8%|▊ | 192MB / 2.35GB  Processing Files (0 / 8) : 26%|██▌ | 5.76GB / 22.2GB, 538MB/s New Data Upload : 52%|█████▏ | 1.53GB / 2.95GB, 128MB/s  ...nts/optim_045000_rank5.pt: 33%|███▎ | 809MB / 2.49GB  ...nts/optim_045000_rank4.pt: 32%|███▏ | 788MB / 2.49GB  ...nts/optim_045000_rank0.pt: 33%|███▎ | 833MB / 2.49GB  ...nts/optim_045000_rank1.pt: 33%|███▎ | 822MB / 2.49GB  ...nts/optim_045000_rank3.pt: 33%|███▎ | 813MB / 2.49GB  ...nts/optim_045000_rank6.pt: 33%|███▎ | 816MB / 2.49GB  ...nts/optim_045000_rank7.pt: 31%|███ | 767MB / 2.49GB  ...eckpoints/model_045000.pt: 8%|▊ | 195MB / 2.35GB  Processing Files (0 / 8) : 26%|██▋ | 5.84GB / 22.2GB, 544MB/s New Data Upload : 51%|█████▏ | 1.55GB / 3.02GB, 128MB/s  ...nts/optim_045000_rank5.pt: 33%|███▎ | 832MB / 2.49GB  ...nts/optim_045000_rank4.pt: 33%|███▎ | 812MB / 2.49GB  ...nts/optim_045000_rank0.pt: 34%|███▍ | 857MB / 2.49GB  ...nts/optim_045000_rank1.pt: 34%|███▍ | 845MB / 2.49GB  ...nts/optim_045000_rank3.pt: 34%|███▎ | 836MB / 2.49GB  ...nts/optim_045000_rank6.pt: 34%|███▎ | 839MB / 2.49GB  ...nts/optim_045000_rank7.pt: 32%|███▏ | 790MB / 2.49GB  ...eckpoints/model_045000.pt: 9%|▊ | 201MB / 2.35GB  Processing Files (0 / 8) : 27%|██▋ | 6.01GB / 22.2GB, 558MB/s New Data Upload : 53%|█████▎ | 1.59GB / 3.02GB, 131MB/s  ...nts/optim_045000_rank5.pt: 34%|███▍ | 850MB / 2.49GB  ...nts/optim_045000_rank4.pt: 33%|███▎ | 830MB / 2.49GB  ...nts/optim_045000_rank0.pt: 35%|███▌ | 875MB / 2.49GB  ...nts/optim_045000_rank1.pt: 35%|███▍ | 863MB / 2.49GB  ...nts/optim_045000_rank3.pt: 34%|███▍ | 856MB / 2.49GB  ...nts/optim_045000_rank6.pt: 35%|███▍ | 858MB / 2.49GB  ...nts/optim_045000_rank7.pt: 33%|███▎ | 808MB / 2.49GB  ...eckpoints/model_045000.pt: 9%|▉ | 209MB / 2.35GB  Processing Files (0 / 8) : 28%|██▊ | 6.15GB / 22.2GB, 568MB/s New Data Upload : 53%|█████▎ | 1.63GB / 3.09GB, 133MB/s  ...nts/optim_045000_rank5.pt: 35%|███▍ | 869MB / 2.49GB  ...nts/optim_045000_rank4.pt: 34%|███▍ | 845MB / 2.49GB  ...nts/optim_045000_rank0.pt: 36%|███▌ | 895MB / 2.49GB  ...nts/optim_045000_rank1.pt: 35%|███▌ | 882MB / 2.49GB  ...nts/optim_045000_rank3.pt: 35%|███▌ | 875MB / 2.49GB  ...nts/optim_045000_rank6.pt: 35%|███▌ | 875MB / 2.49GB  ...nts/optim_045000_rank7.pt: 33%|███▎ | 826MB / 2.49GB  ...eckpoints/model_045000.pt: 9%|▉ | 213MB / 2.35GB  Processing Files (0 / 8) : 28%|██▊ | 6.28GB / 22.2GB, 576MB/s New Data Upload : 54%|█████▍ | 1.66GB / 3.09GB, 134MB/s  ...nts/optim_045000_rank5.pt: 36%|███▌ | 884MB / 2.49GB  ...nts/optim_045000_rank4.pt: 35%|███▍ | 863MB / 2.49GB  ...nts/optim_045000_rank0.pt: 37%|███▋ | 910MB / 2.49GB  ...nts/optim_045000_rank1.pt: 36%|███▌ | 896MB / 2.49GB  ...nts/optim_045000_rank3.pt: 36%|███▌ | 891MB / 2.49GB  ...nts/optim_045000_rank6.pt: 36%|███▌ | 891MB / 2.49GB  ...nts/optim_045000_rank7.pt: 34%|███▍ | 840MB / 2.49GB  ...eckpoints/model_045000.pt: 9%|▉ | 217MB / 2.35GB  Processing Files (0 / 8) : 29%|██▊ | 6.39GB / 22.2GB, 581MB/s New Data Upload : 55%|█████▍ | 1.69GB / 3.09GB, 135MB/s  ...nts/optim_045000_rank5.pt: 36%|███▌ | 900MB / 2.49GB  ...nts/optim_045000_rank4.pt: 35%|███▌ | 878MB / 2.49GB  ...nts/optim_045000_rank0.pt: 37%|███▋ | 925MB / 2.49GB  ...nts/optim_045000_rank1.pt: 37%|███▋ | 911MB / 2.49GB  ...nts/optim_045000_rank3.pt: 36%|███▋ | 906MB / 2.49GB  ...nts/optim_045000_rank6.pt: 36%|███▋ | 905MB / 2.49GB  ...nts/optim_045000_rank7.pt: 34%|███▍ | 856MB / 2.49GB  ...eckpoints/model_045000.pt: 9%|▉ | 221MB / 2.35GB  Processing Files (0 / 8) : 29%|██▉ | 6.50GB / 22.2GB, 586MB/s New Data Upload : 56%|█████▌ | 1.71GB / 3.09GB, 135MB/s  ...nts/optim_045000_rank5.pt: 37%|███▋ | 919MB / 2.49GB  ...nts/optim_045000_rank4.pt: 36%|███▌ | 900MB / 2.49GB  ...nts/optim_045000_rank0.pt: 38%|███▊ | 946MB / 2.49GB  ...nts/optim_045000_rank1.pt: 37%|███▋ | 929MB / 2.49GB  ...nts/optim_045000_rank3.pt: 37%|███▋ | 929MB / 2.49GB  ...nts/optim_045000_rank6.pt: 37%|███▋ | 925MB / 2.49GB  ...nts/optim_045000_rank7.pt: 35%|███▌ | 875MB / 2.49GB  ...eckpoints/model_045000.pt: 10%|▉ | 227MB / 2.35GB  Processing Files (0 / 8) : 30%|██▉ | 6.65GB / 22.2GB, 593MB/s New Data Upload : 56%|█████▌ | 1.75GB / 3.15GB, 137MB/s  ...nts/optim_045000_rank5.pt: 38%|███▊ | 939MB / 2.49GB  ...nts/optim_045000_rank4.pt: 37%|███▋ | 923MB / 2.49GB  ...nts/optim_045000_rank0.pt: 39%|███▉ | 966MB / 2.49GB  ...nts/optim_045000_rank1.pt: 38%|███▊ | 949MB / 2.49GB  ...nts/optim_045000_rank3.pt: 38%|███▊ | 949MB / 2.49GB  ...nts/optim_045000_rank6.pt: 38%|███▊ | 945MB / 2.49GB  ...nts/optim_045000_rank7.pt: 36%|███▌ | 897MB / 2.49GB  ...eckpoints/model_045000.pt: 10%|▉ | 232MB / 2.35GB  Processing Files (0 / 8) : 31%|███ | 6.80GB / 22.2GB, 600MB/s New Data Upload : 57%|█████▋ | 1.79GB / 3.15GB, 139MB/s  ...nts/optim_045000_rank5.pt: 39%|███▊ | 957MB / 2.49GB  ...nts/optim_045000_rank4.pt: 38%|███▊ | 940MB / 2.49GB  ...nts/optim_045000_rank0.pt: 40%|███▉ | 986MB / 2.49GB  ...nts/optim_045000_rank1.pt: 39%|███▉ | 969MB / 2.49GB  ...nts/optim_045000_rank3.pt: 39%|███▉ | 969MB / 2.49GB  ...nts/optim_045000_rank6.pt: 39%|███▊ | 963MB / 2.49GB  ...nts/optim_045000_rank7.pt: 37%|███▋ | 913MB / 2.49GB  ...eckpoints/model_045000.pt: 10%|█ | 236MB / 2.35GB  Processing Files (0 / 8) : 31%|███ | 6.93GB / 22.2GB, 606MB/s New Data Upload : 55%|█████▌ | 1.82GB / 3.29GB, 140MB/s  ...nts/optim_045000_rank5.pt: 39%|███▉ | 971MB / 2.49GB  ...nts/optim_045000_rank4.pt: 39%|███▊ | 959MB / 2.49GB  ...nts/optim_045000_rank0.pt: 40%|████ | 1.00GB / 2.49GB  ...nts/optim_045000_rank1.pt: 39%|███▉ | 982MB / 2.49GB  ...nts/optim_045000_rank3.pt: 39%|███▉ | 982MB / 2.49GB  ...nts/optim_045000_rank6.pt: 39%|███▉ | 976MB / 2.49GB  ...nts/optim_045000_rank7.pt: 37%|███▋ | 926MB / 2.49GB  ...eckpoints/model_045000.pt: 10%|█ | 238MB / 2.35GB  Processing Files (0 / 8) : 32%|███▏ | 7.03GB / 22.2GB, 606MB/s New Data Upload : 56%|█████▋ | 1.85GB / 3.29GB, 140MB/s  ...nts/optim_045000_rank5.pt: 40%|███▉ | 987MB / 2.49GB  ...nts/optim_045000_rank4.pt: 39%|███▉ | 973MB / 2.49GB  ...nts/optim_045000_rank0.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_045000_rank1.pt: 40%|████ | 998MB / 2.49GB  ...nts/optim_045000_rank3.pt: 40%|████ | 998MB / 2.49GB  ...nts/optim_045000_rank6.pt: 40%|███▉ | 993MB / 2.49GB  ...nts/optim_045000_rank7.pt: 38%|███▊ | 943MB / 2.49GB  ...eckpoints/model_045000.pt: 10%|█ | 243MB / 2.35GB  Processing Files (0 / 8) : 32%|███▏ | 7.15GB / 22.2GB, 606MB/s New Data Upload : 56%|█████▌ | 1.88GB / 3.35GB, 141MB/s  ...nts/optim_045000_rank5.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_045000_rank4.pt: 40%|████ | 997MB / 2.49GB  ...nts/optim_045000_rank0.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_045000_rank1.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_045000_rank3.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_045000_rank6.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_045000_rank7.pt: 39%|███▉ | 963MB / 2.49GB  ...eckpoints/model_045000.pt: 11%|█ | 247MB / 2.35GB  Processing Files (0 / 8) : 33%|███▎ | 7.32GB / 22.2GB, 610MB/s New Data Upload : 54%|█████▍ | 1.92GB / 3.55GB, 142MB/s  ...nts/optim_045000_rank5.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_045000_rank4.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_045000_rank0.pt: 42%|████▏ | 1.06GB / 2.49GB  ...nts/optim_045000_rank1.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_045000_rank3.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_045000_rank6.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_045000_rank7.pt: 40%|███▉ | 982MB / 2.49GB  ...eckpoints/model_045000.pt: 11%|█ | 250MB / 2.35GB  Processing Files (0 / 8) : 33%|███▎ | 7.44GB / 22.2GB, 612MB/s New Data Upload : 55%|█████▍ | 1.95GB / 3.55GB, 143MB/s  ...nts/optim_045000_rank5.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_045000_rank4.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_045000_rank0.pt: 44%|████▎ | 1.08GB / 2.49GB  ...nts/optim_045000_rank1.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_045000_rank3.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_045000_rank6.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_045000_rank7.pt: 40%|████ | 1.00GB / 2.49GB  ...eckpoints/model_045000.pt: 11%|█ | 256MB / 2.35GB  Processing Files (0 / 8) : 34%|███▍ | 7.61GB / 22.2GB, 615MB/s New Data Upload : 55%|█████▌ | 2.00GB / 3.62GB, 144MB/s  ...nts/optim_045000_rank5.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_045000_rank4.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_045000_rank0.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_045000_rank1.pt: 44%|████▍ | 1.09GB / 2.49GB  ...nts/optim_045000_rank3.pt: 44%|████▎ | 1.08GB / 2.49GB  ...nts/optim_045000_rank6.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_045000_rank7.pt: 41%|████ | 1.02GB / 2.49GB  ...eckpoints/model_045000.pt: 11%|█ | 263MB / 2.35GB  Processing Files (0 / 8) : 35%|███▍ | 7.77GB / 22.2GB, 620MB/s New Data Upload : 56%|█████▋ | 2.04GB / 3.62GB, 146MB/s  ...nts/optim_045000_rank5.pt: 44%|████▎ | 1.09GB / 2.49GB  ...nts/optim_045000_rank4.pt: 44%|████▎ | 1.09GB / 2.49GB  ...nts/optim_045000_rank0.pt: 45%|████▌ | 1.12GB / 2.49GB  ...nts/optim_045000_rank1.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_045000_rank3.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_045000_rank6.pt: 44%|████▍ | 1.09GB / 2.49GB  ...nts/optim_045000_rank7.pt: 42%|████▏ | 1.04GB / 2.49GB  ...eckpoints/model_045000.pt: 11%|█▏ | 270MB / 2.35GB  Processing Files (0 / 8) : 36%|███▌ | 7.91GB / 22.2GB, 623MB/s New Data Upload : 57%|█████▋ | 2.07GB / 3.62GB, 147MB/s  ...nts/optim_045000_rank5.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_045000_rank4.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_045000_rank0.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_045000_rank1.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_045000_rank3.pt: 45%|████▌ | 1.13GB / 2.49GB  ...nts/optim_045000_rank6.pt: 45%|████▍ | 1.12GB / 2.49GB  ...nts/optim_045000_rank7.pt: 43%|████▎ | 1.07GB / 2.49GB  ...eckpoints/model_045000.pt: 12%|█▏ | 283MB / 2.35GB  Processing Files (0 / 8) : 36%|███▋ | 8.10GB / 22.2GB, 635MB/s New Data Upload : 58%|█████▊ | 2.13GB / 3.69GB, 151MB/s  ...nts/optim_045000_rank5.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_045000_rank4.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_045000_rank0.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_045000_rank1.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_045000_rank3.pt: 46%|████▋ | 1.15GB / 2.49GB  ...nts/optim_045000_rank6.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_045000_rank7.pt: 44%|████▍ | 1.09GB / 2.49GB  ...eckpoints/model_045000.pt: 12%|█▏ | 290MB / 2.35GB  Processing Files (0 / 8) : 37%|███▋ | 8.27GB / 22.2GB, 637MB/s New Data Upload : 59%|█████▉ | 2.17GB / 3.69GB, 152MB/s  ...nts/optim_045000_rank5.pt: 46%|████▋ | 1.15GB / 2.49GB  ...nts/optim_045000_rank4.pt: 46%|████▋ | 1.15GB / 2.49GB  ...nts/optim_045000_rank0.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_045000_rank1.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_045000_rank3.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_045000_rank6.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_045000_rank7.pt: 45%|████▍ | 1.11GB / 2.49GB  ...eckpoints/model_045000.pt: 13%|█▎ | 302MB / 2.35GB  Processing Files (0 / 8) : 38%|███▊ | 8.42GB / 22.2GB, 644MB/s New Data Upload : 59%|█████▉ | 2.21GB / 3.76GB, 154MB/s  ...nts/optim_045000_rank5.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_045000_rank4.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_045000_rank0.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_045000_rank1.pt: 49%|████▊ | 1.21GB / 2.49GB  ...nts/optim_045000_rank3.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_045000_rank6.pt: 47%|████▋ | 1.18GB / 2.49GB  ...nts/optim_045000_rank7.pt: 45%|████▌ | 1.13GB / 2.49GB  ...eckpoints/model_045000.pt: 13%|█▎ | 308MB / 2.35GB  Processing Files (0 / 8) : 39%|███▊ | 8.57GB / 22.2GB, 646MB/s New Data Upload : 60%|█████▉ | 2.25GB / 3.76GB, 155MB/s  ...nts/optim_045000_rank5.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_045000_rank4.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_045000_rank0.pt: 50%|████▉ | 1.23GB / 2.49GB  ...nts/optim_045000_rank1.pt: 50%|████▉ | 1.23GB / 2.49GB  ...nts/optim_045000_rank3.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_045000_rank6.pt: 49%|████▊ | 1.21GB / 2.49GB  ...nts/optim_045000_rank7.pt: 46%|████▋ | 1.15GB / 2.49GB  ...eckpoints/model_045000.pt: 13%|█▎ | 317MB / 2.35GB  Processing Files (0 / 8) : 39%|███▉ | 8.75GB / 22.2GB, 658MB/s New Data Upload : 61%|██████ | 2.30GB / 3.76GB, 158MB/s  ...nts/optim_045000_rank5.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_045000_rank4.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_045000_rank0.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_045000_rank1.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_045000_rank3.pt: 50%|████▉ | 1.23GB / 2.49GB  ...nts/optim_045000_rank6.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_045000_rank7.pt: 47%|████▋ | 1.17GB / 2.49GB  ...eckpoints/model_045000.pt: 14%|█▍ | 323MB / 2.35GB  Processing Files (0 / 8) : 40%|███▉ | 8.89GB / 22.2GB, 659MB/s New Data Upload : 61%|██████ | 2.34GB / 3.82GB, 159MB/s  ...nts/optim_045000_rank5.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_045000_rank4.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_045000_rank0.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_045000_rank1.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_045000_rank3.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_045000_rank6.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_045000_rank7.pt: 48%|████▊ | 1.20GB / 2.49GB  ...eckpoints/model_045000.pt: 14%|█▍ | 335MB / 2.35GB  Processing Files (0 / 8) : 41%|████ | 9.07GB / 22.2GB, 670MB/s New Data Upload : 61%|██████▏ | 2.38GB / 3.89GB, 162MB/s  ...nts/optim_045000_rank5.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_045000_rank4.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_045000_rank0.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_045000_rank1.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_045000_rank3.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_045000_rank6.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_045000_rank7.pt: 49%|████▉ | 1.21GB / 2.49GB  ...eckpoints/model_045000.pt: 14%|█▍ | 337MB / 2.35GB  Processing Files (0 / 8) : 41%|████▏ | 9.22GB / 22.2GB, 674MB/s New Data Upload : 61%|██████ | 2.42GB / 3.96GB, 163MB/s  ...nts/optim_045000_rank5.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_045000_rank4.pt: 52%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_045000_rank0.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_045000_rank1.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_045000_rank3.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_045000_rank6.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_045000_rank7.pt: 50%|████▉ | 1.23GB / 2.49GB  ...eckpoints/model_045000.pt: 15%|█▍ | 349MB / 2.35GB  Processing Files (0 / 8) : 42%|████▏ | 9.37GB / 22.2GB, 679MB/s New Data Upload : 61%|██████▏ | 2.47GB / 4.02GB, 165MB/s  ...nts/optim_045000_rank5.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_045000_rank4.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_045000_rank0.pt: 53%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_045000_rank1.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_045000_rank3.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_045000_rank6.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_045000_rank7.pt: 51%|█████ | 1.26GB / 2.49GB  ...eckpoints/model_045000.pt: 15%|█▌ | 358MB / 2.35GB  Processing Files (0 / 8) : 43%|████▎ | 9.53GB / 22.2GB, 684MB/s New Data Upload : 60%|██████ | 2.51GB / 4.16GB, 167MB/s  ...nts/optim_045000_rank5.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_045000_rank4.pt: 54%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_045000_rank0.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_045000_rank1.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_045000_rank3.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_045000_rank6.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_045000_rank7.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...eckpoints/model_045000.pt: 16%|█▌ | 369MB / 2.35GB  Processing Files (0 / 8) : 44%|████▎ | 9.70GB / 22.2GB, 693MB/s New Data Upload : 62%|██████▏ | 2.56GB / 4.16GB, 170MB/s  ...nts/optim_045000_rank5.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_045000_rank4.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_045000_rank0.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_045000_rank1.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_045000_rank3.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_045000_rank6.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_045000_rank7.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...eckpoints/model_045000.pt: 16%|█▌ | 375MB / 2.35GB  Processing Files (0 / 8) : 44%|████▍ | 9.87GB / 22.2GB, 705MB/s New Data Upload : 63%|██████▎ | 2.60GB / 4.16GB, 172MB/s  ...nts/optim_045000_rank5.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_045000_rank4.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_045000_rank0.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_045000_rank1.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_045000_rank3.pt: 56%|█████▌ | 1.40GB / 2.49GB  ...nts/optim_045000_rank6.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_045000_rank7.pt: 53%|█████▎ | 1.33GB / 2.49GB  ...eckpoints/model_045000.pt: 16%|█▌ | 378MB / 2.35GB  Processing Files (0 / 8) : 45%|████▌ | 10.1GB / 22.2GB, 715MB/s New Data Upload : 63%|██████▎ | 2.65GB / 4.23GB, 174MB/s  ...nts/optim_045000_rank5.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_045000_rank4.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_045000_rank0.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_045000_rank1.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_045000_rank3.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_045000_rank6.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_045000_rank7.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...eckpoints/model_045000.pt: 16%|█▋ | 385MB / 2.35GB  Processing Files (0 / 8) : 46%|████▋ | 10.3GB / 22.2GB, 727MB/s New Data Upload : 63%|██████▎ | 2.70GB / 4.29GB, 177MB/s  ...nts/optim_045000_rank5.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_045000_rank4.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_045000_rank0.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_045000_rank1.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 59%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_045000_rank6.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_045000_rank7.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...eckpoints/model_045000.pt: 17%|█▋ | 388MB / 2.35GB  Processing Files (0 / 8) : 47%|████▋ | 10.5GB / 22.2GB, 739MB/s New Data Upload : 63%|██████▎ | 2.74GB / 4.36GB, 179MB/s  ...nts/optim_045000_rank5.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_045000_rank4.pt: 59%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_045000_rank3.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...eckpoints/model_045000.pt: 17%|█▋ | 394MB / 2.35GB  Processing Files (0 / 8) : 48%|████▊ | 10.7GB / 22.2GB, 753MB/s New Data Upload : 64%|██████▍ | 2.79GB / 4.36GB, 182MB/s  ...nts/optim_045000_rank5.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_045000_rank0.pt: 60%|██████ | 1.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_045000_rank3.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_045000_rank6.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_045000_rank7.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...eckpoints/model_045000.pt: 17%|█▋ | 398MB / 2.35GB  Processing Files (0 / 8) : 49%|████▉ | 10.9GB / 22.2GB, 762MB/s New Data Upload : 65%|██████▍ | 2.83GB / 4.36GB, 183MB/s  ...nts/optim_045000_rank5.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_045000_rank4.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_045000_rank0.pt: 61%|██████ | 1.52GB / 2.49GB  ...nts/optim_045000_rank1.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_045000_rank3.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_045000_rank6.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_045000_rank7.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...eckpoints/model_045000.pt: 17%|█▋ | 408MB / 2.35GB  Processing Files (0 / 8) : 50%|████▉ | 11.1GB / 22.2GB, 774MB/s New Data Upload : 65%|██████▍ | 2.88GB / 4.43GB, 186MB/s  ...nts/optim_045000_rank5.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_045000_rank4.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_045000_rank0.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_045000_rank1.pt: 64%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_045000_rank3.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_045000_rank6.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_045000_rank7.pt: 60%|██████ | 1.50GB / 2.49GB  ...eckpoints/model_045000.pt: 18%|█▊ | 416MB / 2.35GB  Processing Files (0 / 8) : 51%|█████ | 11.3GB / 22.2GB, 784MB/s New Data Upload : 66%|██████▌ | 2.93GB / 4.43GB, 189MB/s  ...nts/optim_045000_rank5.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_045000_rank4.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_045000_rank0.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_045000_rank1.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_045000_rank3.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_045000_rank6.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_045000_rank7.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...eckpoints/model_045000.pt: 18%|█▊ | 423MB / 2.35GB  Processing Files (0 / 8) : 52%|█████▏ | 11.5GB / 22.2GB, 793MB/s New Data Upload : 67%|██████▋ | 2.97GB / 4.43GB, 191MB/s  ...nts/optim_045000_rank5.pt: 64%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_045000_rank4.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_045000_rank0.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_045000_rank1.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_045000_rank3.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_045000_rank6.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_045000_rank7.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...eckpoints/model_045000.pt: 18%|█▊ | 426MB / 2.35GB  Processing Files (0 / 8) : 52%|█████▏ | 11.6GB / 22.2GB, 790MB/s New Data Upload : 65%|██████▌ | 3.01GB / 4.61GB, 191MB/s  ...nts/optim_045000_rank2.pt: 4%|▎ | 87.9MB / 2.49GB  ...nts/optim_045000_rank5.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_045000_rank4.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_045000_rank0.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_045000_rank1.pt: 67%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_045000_rank3.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_045000_rank6.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_045000_rank7.pt: 64%|██████▎ | 1.58GB / 2.49GB  ...eckpoints/model_045000.pt: 18%|█▊ | 434MB / 2.35GB  ...nts/optim_045000_rank2.pt: 4%|▎ | 87.9MB / 2.49GB  Processing Files (0 / 9) : 54%|█████▎ | 11.9GB / 22.2GB, 809MB/s New Data Upload : 66%|██████▋ | 3.06GB / 4.61GB, 194MB/s  ...nts/optim_045000_rank5.pt: 65%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_045000_rank4.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_045000_rank0.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_045000_rank1.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_045000_rank3.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_045000_rank6.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_045000_rank7.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...eckpoints/model_045000.pt: 19%|█▉ | 442MB / 2.35GB  ...nts/optim_045000_rank2.pt: 9%|▊ | 215MB / 2.49GB  Processing Files (0 / 9) : 55%|█████▍ | 12.2GB / 22.2GB, 814MB/s New Data Upload : 66%|██████▋ | 3.09GB / 4.67GB, 193MB/s  ...nts/optim_045000_rank5.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_045000_rank4.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_045000_rank0.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_045000_rank1.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_045000_rank3.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_045000_rank6.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_045000_rank7.pt: 65%|██████▌ | 1.63GB / 2.49GB  ...eckpoints/model_045000.pt: 19%|█▉ | 451MB / 2.35GB  ...nts/optim_045000_rank2.pt: 14%|█▎ | 340MB / 2.49GB  Processing Files (0 / 9) : 56%|█████▋ | 12.5GB / 22.2GB, 838MB/s New Data Upload : 68%|██████▊ | 3.16GB / 4.67GB, 197MB/s  ...nts/optim_045000_rank5.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_045000_rank4.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_045000_rank0.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_045000_rank1.pt: 69%|██████▊ | 1.71GB / 2.49GB  ...nts/optim_045000_rank3.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_045000_rank6.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_045000_rank7.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...eckpoints/model_045000.pt: 20%|█▉ | 461MB / 2.35GB  ...nts/optim_045000_rank2.pt: 17%|█▋ | 434MB / 2.49GB  Processing Files (0 / 9) : 57%|█████▋ | 12.8GB / 22.2GB, 845MB/s New Data Upload : 68%|██████▊ | 3.20GB / 4.73GB, 198MB/s  ...nts/optim_045000_rank5.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_045000_rank4.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_045000_rank0.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_045000_rank1.pt: 69%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_045000_rank3.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_045000_rank6.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_045000_rank7.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...eckpoints/model_045000.pt: 20%|█▉ | 467MB / 2.35GB  ...nts/optim_045000_rank2.pt: 19%|█▉ | 475MB / 2.49GB  Processing Files (0 / 9) : 58%|█████▊ | 13.0GB / 22.2GB, 849MB/s New Data Upload : 68%|██████▊ | 3.24GB / 4.73GB, 199MB/s  ...nts/optim_045000_rank5.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_045000_rank4.pt: 71%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_045000_rank0.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_045000_rank1.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_045000_rank3.pt: 71%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_045000_rank6.pt: 71%|███████▏ | 1.77GB / 2.49GB  ...nts/optim_045000_rank7.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...eckpoints/model_045000.pt: 20%|██ | 481MB / 2.35GB  ...nts/optim_045000_rank2.pt: 26%|██▌ | 652MB / 2.49GB  Processing Files (0 / 9) : 60%|██████ | 13.4GB / 22.2GB, 872MB/s New Data Upload : 70%|██████▉ | 3.30GB / 4.73GB, 200MB/s  ...nts/optim_045000_rank5.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_045000_rank4.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_045000_rank0.pt: 71%|███████▏ | 1.77GB / 2.49GB  ...nts/optim_045000_rank1.pt: 71%|███████▏ | 1.77GB / 2.49GB  ...nts/optim_045000_rank3.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_045000_rank6.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_045000_rank7.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...eckpoints/model_045000.pt: 21%|██ | 488MB / 2.35GB  ...nts/optim_045000_rank2.pt: 34%|███▍ | 848MB / 2.49GB  Processing Files (0 / 9) : 62%|██████▏ | 13.8GB / 22.2GB, 897MB/s New Data Upload : 70%|██████▉ | 3.34GB / 4.80GB, 203MB/s  ...nts/optim_045000_rank5.pt: 71%|███████▏ | 1.77GB / 2.49GB  ...nts/optim_045000_rank4.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_045000_rank0.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_045000_rank1.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_045000_rank3.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_045000_rank6.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_045000_rank7.pt: 70%|███████ | 1.74GB / 2.49GB  ...eckpoints/model_045000.pt: 22%|██▏ | 507MB / 2.35GB  ...nts/optim_045000_rank2.pt: 39%|███▉ | 978MB / 2.49GB  Processing Files (0 / 9) : 63%|██████▎ | 14.1GB / 22.2GB, 912MB/s New Data Upload : 71%|███████ | 3.39GB / 4.80GB, 204MB/s  ...nts/optim_045000_rank5.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_045000_rank4.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_045000_rank0.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_045000_rank1.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_045000_rank3.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_045000_rank6.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_045000_rank7.pt: 71%|███████ | 1.76GB / 2.49GB  ...eckpoints/model_045000.pt: 22%|██▏ | 514MB / 2.35GB  ...nts/optim_045000_rank2.pt: 43%|████▎ | 1.08GB / 2.49GB  Processing Files (0 / 9) : 64%|██████▍ | 14.3GB / 22.2GB, 924MB/s New Data Upload : 70%|███████ | 3.43GB / 4.87GB, 205MB/s  ...nts/optim_045000_rank5.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_045000_rank4.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_045000_rank0.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_045000_rank1.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_045000_rank3.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_045000_rank6.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_045000_rank7.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...eckpoints/model_045000.pt: 23%|██▎ | 530MB / 2.35GB  ...nts/optim_045000_rank2.pt: 46%|████▌ | 1.14GB / 2.49GB  Processing Files (0 / 9) : 65%|██████▌ | 14.6GB / 22.2GB, 925MB/s New Data Upload : 72%|███████▏ | 3.49GB / 4.87GB, 206MB/s  ...nts/optim_045000_rank5.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_045000_rank4.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_045000_rank0.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_045000_rank1.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_045000_rank3.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_045000_rank6.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_045000_rank7.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...eckpoints/model_045000.pt: 23%|██▎ | 539MB / 2.35GB  ...nts/optim_045000_rank2.pt: 49%|████▉ | 1.21GB / 2.49GB  Processing Files (0 / 9) : 67%|██████▋ | 14.8GB / 22.2GB, 935MB/s New Data Upload : 72%|███████▏ | 3.53GB / 4.94GB, 208MB/s  ...nts/optim_045000_rank5.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_045000_rank4.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_045000_rank0.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_045000_rank1.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_045000_rank3.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_045000_rank6.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_045000_rank7.pt: 74%|███████▎ | 1.83GB / 2.49GB  ...eckpoints/model_045000.pt: 24%|██▎ | 556MB / 2.35GB  ...nts/optim_045000_rank2.pt: 52%|█████▏ | 1.28GB / 2.49GB  Processing Files (0 / 9) : 68%|██████▊ | 15.1GB / 22.2GB, 947MB/s New Data Upload : 72%|███████▏ | 3.59GB / 5.00GB, 210MB/s  ...nts/optim_045000_rank5.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_045000_rank4.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_045000_rank0.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_045000_rank1.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_045000_rank3.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_045000_rank6.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_045000_rank7.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...eckpoints/model_045000.pt: 24%|██▍ | 570MB / 2.35GB  ...nts/optim_045000_rank2.pt: 56%|█████▌ | 1.39GB / 2.49GB  Processing Files (0 / 9) : 69%|██████▉ | 15.3GB / 22.2GB, 955MB/s New Data Upload : 73%|███████▎ | 3.63GB / 5.00GB, 210MB/s  ...nts/optim_045000_rank5.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_045000_rank4.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_045000_rank0.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_045000_rank1.pt: 77%|███████▋ | 1.93GB / 2.49GB  ...nts/optim_045000_rank3.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_045000_rank6.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_045000_rank7.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...eckpoints/model_045000.pt: 25%|██▍ | 579MB / 2.35GB  ...nts/optim_045000_rank2.pt: 59%|█████▊ | 1.46GB / 2.49GB  Processing Files (0 / 9) : 70%|██████▉ | 15.5GB / 22.2GB, 958MB/s New Data Upload : 72%|███████▏ | 3.67GB / 5.07GB, 210MB/s  ...nts/optim_045000_rank5.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_045000_rank4.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_045000_rank0.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_045000_rank1.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_045000_rank3.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_045000_rank6.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_045000_rank7.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...eckpoints/model_045000.pt: 25%|██▍ | 587MB / 2.35GB  ...nts/optim_045000_rank2.pt: 62%|██████▏ | 1.54GB / 2.49GB  Processing Files (0 / 9) : 71%|███████ | 15.8GB / 22.2GB, 973MB/s New Data Upload : 73%|███████▎ | 3.71GB / 5.07GB, 212MB/s  ...nts/optim_045000_rank5.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_045000_rank4.pt: 80%|███████▉ | 1.99GB / 2.49GB  ...nts/optim_045000_rank0.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_045000_rank1.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_045000_rank3.pt: 81%|████████ | 2.00GB / 2.49GB  ...nts/optim_045000_rank6.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_045000_rank7.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...eckpoints/model_045000.pt: 26%|██▌ | 599MB / 2.35GB  ...nts/optim_045000_rank2.pt: 67%|██████▋ | 1.66GB / 2.49GB  Processing Files (0 / 9) : 72%|███████▏ | 16.0GB / 22.2GB, 982MB/s New Data Upload : 73%|███████▎ | 3.76GB / 5.14GB, 212MB/s  ...nts/optim_045000_rank5.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_045000_rank4.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_045000_rank0.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_045000_rank1.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_045000_rank3.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_045000_rank6.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_045000_rank7.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...eckpoints/model_045000.pt: 26%|██▌ | 616MB / 2.35GB  ...nts/optim_045000_rank2.pt: 70%|███████ | 1.75GB / 2.49GB  Processing Files (0 / 9) : 73%|███████▎ | 16.3GB / 22.2GB, 993MB/s New Data Upload : 73%|███████▎ | 3.81GB / 5.20GB, 214MB/s  ...nts/optim_045000_rank5.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_045000_rank4.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_045000_rank0.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_045000_rank1.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_045000_rank3.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_045000_rank6.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_045000_rank7.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...eckpoints/model_045000.pt: 27%|██▋ | 634MB / 2.35GB  ...nts/optim_045000_rank2.pt: 78%|███████▊ | 1.93GB / 2.49GB  Processing Files (0 / 9) : 75%|███████▍ | 16.6GB / 22.2GB, 1.01GB/s New Data Upload : 73%|███████▎ | 3.85GB / 5.27GB, 215MB/s  ...nts/optim_045000_rank5.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_045000_rank4.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_045000_rank0.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_045000_rank1.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_045000_rank3.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_045000_rank6.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_045000_rank7.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...eckpoints/model_045000.pt: 28%|██▊ | 649MB / 2.35GB  ...nts/optim_045000_rank2.pt: 81%|████████▏ | 2.02GB / 2.49GB  Processing Files (0 / 9) : 76%|███████▌ | 16.8GB / 22.2GB, 1.02GB/s New Data Upload : 74%|███████▍ | 3.90GB / 5.27GB, 217MB/s  ...nts/optim_045000_rank5.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_045000_rank4.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_045000_rank0.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_045000_rank1.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_045000_rank3.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_045000_rank6.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_045000_rank7.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...eckpoints/model_045000.pt: 28%|██▊ | 658MB / 2.35GB  ...nts/optim_045000_rank2.pt: 83%|████████▎ | 2.07GB / 2.49GB  Processing Files (0 / 9) : 77%|███████▋ | 17.0GB / 22.2GB, 1.03GB/s New Data Upload : 74%|███████▍ | 3.94GB / 5.34GB, 218MB/s  ...nts/optim_045000_rank5.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_045000_rank4.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_045000_rank0.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_045000_rank1.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_045000_rank3.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_045000_rank6.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_045000_rank7.pt: 79%|███████▉ | 1.98GB / 2.49GB  ...eckpoints/model_045000.pt: 29%|██▉ | 683MB / 2.35GB  ...nts/optim_045000_rank2.pt: 84%|████████▍ | 2.08GB / 2.49GB  Processing Files (0 / 9) : 77%|███████▋ | 17.1GB / 22.2GB, 1.03GB/s New Data Upload : 75%|███████▍ | 3.98GB / 5.34GB, 219MB/s  ...nts/optim_045000_rank5.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_045000_rank4.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_045000_rank0.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_045000_rank1.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_045000_rank3.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_045000_rank6.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_045000_rank7.pt: 80%|████████ | 1.99GB / 2.49GB  ...eckpoints/model_045000.pt: 30%|██▉ | 695MB / 2.35GB  ...nts/optim_045000_rank2.pt: 85%|████████▍ | 2.10GB / 2.49GB  Processing Files (0 / 9) : 78%|███████▊ | 17.3GB / 22.2GB, 1.03GB/s New Data Upload : 74%|███████▍ | 4.03GB / 5.40GB, 219MB/s  ...nts/optim_045000_rank5.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_045000_rank4.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_045000_rank0.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_045000_rank1.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_045000_rank3.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_045000_rank6.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_045000_rank7.pt: 81%|████████ | 2.01GB / 2.49GB  ...eckpoints/model_045000.pt: 31%|███ | 719MB / 2.35GB  ...nts/optim_045000_rank2.pt: 85%|████████▌ | 2.12GB / 2.49GB  Processing Files (0 / 9) : 79%|███████▊ | 17.5GB / 22.2GB, 1.03GB/s New Data Upload : 74%|███████▍ | 4.07GB / 5.47GB, 221MB/s  ...nts/optim_045000_rank5.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_045000_rank4.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_045000_rank0.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_045000_rank1.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_045000_rank3.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_045000_rank6.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_045000_rank7.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...eckpoints/model_045000.pt: 31%|███▏ | 739MB / 2.35GB  ...nts/optim_045000_rank2.pt: 86%|████████▌ | 2.14GB / 2.49GB  Processing Files (0 / 9) : 79%|███████▉ | 17.6GB / 22.2GB, 1.04GB/s New Data Upload : 75%|███████▌ | 4.12GB / 5.47GB, 223MB/s  ...nts/optim_045000_rank5.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_045000_rank4.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_045000_rank0.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_045000_rank1.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_045000_rank3.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_045000_rank6.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_045000_rank7.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...eckpoints/model_045000.pt: 32%|███▏ | 760MB / 2.35GB  ...nts/optim_045000_rank2.pt: 87%|████████▋ | 2.15GB / 2.49GB  Processing Files (0 / 9) : 80%|███████▉ | 17.8GB / 22.2GB, 1.04GB/s New Data Upload : 75%|███████▌ | 4.17GB / 5.54GB, 225MB/s  ...nts/optim_045000_rank5.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_045000_rank4.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_045000_rank0.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_045000_rank1.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_045000_rank3.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_045000_rank6.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_045000_rank7.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...eckpoints/model_045000.pt: 34%|███▍ | 796MB / 2.35GB  ...nts/optim_045000_rank2.pt: 87%|████████▋ | 2.17GB / 2.49GB  Processing Files (0 / 9) : 81%|████████ | 17.9GB / 22.2GB, 1.04GB/s New Data Upload : 75%|███████▌ | 4.22GB / 5.61GB, 226MB/s  ...nts/optim_045000_rank5.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_045000_rank4.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_045000_rank0.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_045000_rank1.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_045000_rank3.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_045000_rank6.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_045000_rank7.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...eckpoints/model_045000.pt: 35%|███▌ | 823MB / 2.35GB  ...nts/optim_045000_rank2.pt: 88%|████████▊ | 2.18GB / 2.49GB  Processing Files (0 / 9) : 81%|████████ | 18.0GB / 22.2GB, 1.04GB/s New Data Upload : 76%|███████▌ | 4.27GB / 5.61GB, 227MB/s  ...nts/optim_045000_rank5.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_045000_rank4.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_045000_rank0.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_045000_rank1.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_045000_rank3.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_045000_rank6.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_045000_rank7.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...eckpoints/model_045000.pt: 36%|███▌ | 850MB / 2.35GB  ...nts/optim_045000_rank2.pt: 88%|████████▊ | 2.19GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.2GB / 22.2GB, 1.04GB/s New Data Upload : 76%|███████▌ | 4.32GB / 5.67GB, 228MB/s  ...nts/optim_045000_rank5.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_045000_rank4.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_045000_rank0.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_045000_rank1.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_045000_rank3.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_045000_rank6.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_045000_rank7.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...eckpoints/model_045000.pt: 38%|███▊ | 883MB / 2.35GB  ...nts/optim_045000_rank2.pt: 89%|████████▉ | 2.21GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.4GB / 22.2GB, 1.04GB/s New Data Upload : 77%|███████▋ | 4.38GB / 5.67GB, 229MB/s  ...nts/optim_045000_rank5.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_045000_rank4.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_045000_rank0.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_045000_rank1.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_045000_rank3.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_045000_rank6.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_045000_rank7.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...eckpoints/model_045000.pt: 39%|███▉ | 913MB / 2.35GB  ...nts/optim_045000_rank2.pt: 89%|████████▉ | 2.22GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.5GB / 22.2GB, 1.03GB/s New Data Upload : 77%|███████▋ | 4.42GB / 5.74GB, 230MB/s  ...nts/optim_045000_rank5.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_045000_rank4.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_045000_rank0.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_045000_rank1.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_045000_rank3.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_045000_rank6.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_045000_rank7.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...eckpoints/model_045000.pt: 40%|████ | 945MB / 2.35GB  ...nts/optim_045000_rank2.pt: 90%|████████▉ | 2.23GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▍ | 18.6GB / 22.2GB, 1.03GB/s New Data Upload : 77%|███████▋ | 4.48GB / 5.81GB, 230MB/s  ...nts/optim_045000_rank5.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_045000_rank4.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_045000_rank0.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_045000_rank1.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_045000_rank3.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_045000_rank6.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_045000_rank7.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...eckpoints/model_045000.pt: 42%|████▏ | 978MB / 2.35GB  ...nts/optim_045000_rank2.pt: 91%|█████████ | 2.25GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▍ | 18.8GB / 22.2GB, 1.03GB/s New Data Upload : 77%|███████▋ | 4.53GB / 5.87GB, 232MB/s  ...nts/optim_045000_rank5.pt: 90%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_045000_rank4.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_045000_rank0.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_045000_rank1.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_045000_rank3.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_045000_rank6.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_045000_rank7.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...eckpoints/model_045000.pt: 43%|████▎ | 1.02GB / 2.35GB  ...nts/optim_045000_rank2.pt: 91%|█████████ | 2.26GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▌ | 18.9GB / 22.2GB, 1.03GB/s New Data Upload : 77%|███████▋ | 4.59GB / 5.94GB, 233MB/s  ...nts/optim_045000_rank5.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_045000_rank4.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_045000_rank0.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_045000_rank1.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_045000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_045000_rank6.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_045000_rank7.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...eckpoints/model_045000.pt: 45%|████▍ | 1.05GB / 2.35GB  ...nts/optim_045000_rank2.pt: 91%|█████████▏| 2.27GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▌ | 19.0GB / 22.2GB, 1.03GB/s New Data Upload : 78%|███████▊ | 4.64GB / 5.94GB, 235MB/s  ...nts/optim_045000_rank5.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_045000_rank4.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_045000_rank0.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_045000_rank1.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_045000_rank3.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_045000_rank6.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_045000_rank7.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...eckpoints/model_045000.pt: 46%|████▌ | 1.07GB / 2.35GB  ...nts/optim_045000_rank2.pt: 92%|█████████▏| 2.29GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▋ | 19.2GB / 22.2GB, 1.02GB/s New Data Upload : 78%|███████▊ | 4.69GB / 6.01GB, 235MB/s  ...nts/optim_045000_rank5.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_045000_rank4.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_045000_rank0.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_045000_rank1.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_045000_rank3.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_045000_rank6.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_045000_rank7.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...eckpoints/model_045000.pt: 47%|████▋ | 1.11GB / 2.35GB  ...nts/optim_045000_rank2.pt: 92%|█████████▏| 2.30GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.3GB / 22.2GB, 1.02GB/s New Data Upload : 78%|███████▊ | 4.74GB / 6.08GB, 236MB/s  ...nts/optim_045000_rank5.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_045000_rank4.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_045000_rank0.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_045000_rank1.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_045000_rank3.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_045000_rank6.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_045000_rank7.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...eckpoints/model_045000.pt: 49%|████▊ | 1.14GB / 2.35GB  ...nts/optim_045000_rank2.pt: 93%|█████████▎| 2.30GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.4GB / 22.2GB, 1.01GB/s New Data Upload : 79%|███████▉ | 4.79GB / 6.08GB, 236MB/s  ...nts/optim_045000_rank5.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_045000_rank4.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_045000_rank0.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_045000_rank1.pt: 92%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_045000_rank3.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_045000_rank6.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_045000_rank7.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...eckpoints/model_045000.pt: 50%|████▉ | 1.17GB / 2.35GB  ...nts/optim_045000_rank2.pt: 93%|█████████▎| 2.31GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.5GB / 22.2GB, 1.01GB/s New Data Upload : 80%|███████▉ | 4.84GB / 6.08GB, 237MB/s  ...nts/optim_045000_rank5.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_045000_rank4.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_045000_rank0.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_045000_rank1.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_045000_rank3.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_045000_rank6.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_045000_rank7.pt: 90%|█████████ | 2.24GB / 2.49GB  ...eckpoints/model_045000.pt: 51%|█████▏ | 1.21GB / 2.35GB  ...nts/optim_045000_rank2.pt: 93%|█████████▎| 2.32GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.6GB / 22.2GB, 1.00GB/s New Data Upload : 79%|███████▉ | 4.88GB / 6.14GB, 237MB/s  ...nts/optim_045000_rank5.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_045000_rank4.pt: 94%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_045000_rank0.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_045000_rank1.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_045000_rank3.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_045000_rank6.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_045000_rank7.pt: 91%|█████████ | 2.25GB / 2.49GB  ...eckpoints/model_045000.pt: 53%|█████▎ | 1.24GB / 2.35GB  ...nts/optim_045000_rank2.pt: 94%|█████████▎| 2.33GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▊ | 19.7GB / 22.2GB, 997MB/s New Data Upload : 80%|████████ | 4.93GB / 6.14GB, 237MB/s  ...nts/optim_045000_rank5.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_045000_rank4.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_045000_rank0.pt: 94%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_045000_rank1.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_045000_rank3.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_045000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_045000_rank7.pt: 91%|█████████ | 2.26GB / 2.49GB  ...eckpoints/model_045000.pt: 54%|█████▍ | 1.28GB / 2.35GB  ...nts/optim_045000_rank2.pt: 94%|█████████▍| 2.33GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▉ | 19.8GB / 22.2GB, 990MB/s New Data Upload : 80%|████████ | 4.98GB / 6.21GB, 237MB/s  ...nts/optim_045000_rank5.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_045000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_045000_rank0.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_045000_rank1.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_045000_rank3.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_045000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_045000_rank7.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...eckpoints/model_045000.pt: 56%|█████▌ | 1.31GB / 2.35GB  ...nts/optim_045000_rank2.pt: 94%|█████████▍| 2.34GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▉ | 19.9GB / 22.2GB, 981MB/s New Data Upload : 81%|████████ | 5.02GB / 6.21GB, 237MB/s  ...nts/optim_045000_rank5.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_045000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_045000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_045000_rank1.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_045000_rank3.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_045000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_045000_rank7.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...eckpoints/model_045000.pt: 57%|█████▋ | 1.33GB / 2.35GB  ...nts/optim_045000_rank2.pt: 94%|█████████▍| 2.34GB / 2.49GB  Processing Files (0 / 9) : 90%|████████▉ | 19.9GB / 22.2GB, 969MB/s New Data Upload : 81%|████████ | 5.06GB / 6.28GB, 236MB/s  ...nts/optim_045000_rank5.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_045000_rank4.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_045000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_045000_rank1.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_045000_rank3.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_045000_rank6.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_045000_rank7.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...eckpoints/model_045000.pt: 59%|█████▊ | 1.37GB / 2.35GB  ...nts/optim_045000_rank2.pt: 94%|█████████▍| 2.35GB / 2.49GB  Processing Files (0 / 9) : 90%|█████████ | 20.0GB / 22.2GB, 956MB/s New Data Upload : 81%|████████▏ | 5.11GB / 6.28GB, 236MB/s  ...nts/optim_045000_rank5.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_045000_rank4.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_045000_rank0.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_045000_rank1.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_045000_rank3.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_045000_rank6.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_045000_rank7.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...eckpoints/model_045000.pt: 60%|█████▉ | 1.40GB / 2.35GB  ...nts/optim_045000_rank2.pt: 95%|█████████▍| 2.36GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.1GB / 22.2GB, 948MB/s New Data Upload : 81%|████████▏ | 5.16GB / 6.34GB, 237MB/s  ...nts/optim_045000_rank5.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_045000_rank4.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_045000_rank0.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_045000_rank1.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_045000_rank3.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_045000_rank6.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_045000_rank7.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...eckpoints/model_045000.pt: 61%|██████ | 1.44GB / 2.35GB  ...nts/optim_045000_rank2.pt: 95%|█████████▍| 2.36GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.2GB / 22.2GB, 932MB/s New Data Upload : 81%|████████▏ | 5.19GB / 6.38GB, 236MB/s  ...nts/optim_045000_rank5.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_045000_rank4.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_045000_rank0.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_045000_rank1.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_045000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_045000_rank6.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_045000_rank7.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...eckpoints/model_045000.pt: 62%|██████▏ | 1.47GB / 2.35GB  ...nts/optim_045000_rank2.pt: 95%|█████████▌| 2.36GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.3GB / 22.2GB, 922MB/s New Data Upload : 82%|████████▏ | 5.24GB / 6.38GB, 236MB/s  ...nts/optim_045000_rank5.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_045000_rank4.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_045000_rank0.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_045000_rank1.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_045000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_045000_rank6.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_045000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...eckpoints/model_045000.pt: 64%|██████▎ | 1.50GB / 2.35GB  ...nts/optim_045000_rank2.pt: 95%|█████████▌| 2.37GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████▏| 20.3GB / 22.2GB, 908MB/s New Data Upload : 83%|████████▎ | 5.28GB / 6.38GB, 235MB/s  ...nts/optim_045000_rank5.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_045000_rank4.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_045000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_045000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_045000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_045000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_045000_rank7.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...eckpoints/model_045000.pt: 65%|██████▌ | 1.53GB / 2.35GB  ...nts/optim_045000_rank2.pt: 95%|█████████▌| 2.37GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.4GB / 22.2GB, 895MB/s New Data Upload : 83%|████████▎ | 5.32GB / 6.38GB, 235MB/s  ...nts/optim_045000_rank5.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_045000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_045000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_045000_rank1.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_045000_rank3.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_045000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_045000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...eckpoints/model_045000.pt: 67%|██████▋ | 1.56GB / 2.35GB  ...nts/optim_045000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.5GB / 22.2GB, 886MB/s New Data Upload : 84%|████████▍ | 5.37GB / 6.38GB, 235MB/s  ...nts/optim_045000_rank5.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_045000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_045000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_045000_rank1.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_045000_rank3.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_045000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_045000_rank7.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...eckpoints/model_045000.pt: 68%|██████▊ | 1.60GB / 2.35GB  ...nts/optim_045000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.6GB / 22.2GB, 875MB/s New Data Upload : 85%|████████▍ | 5.42GB / 6.38GB, 236MB/s  ...nts/optim_045000_rank5.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_045000_rank4.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_045000_rank0.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_045000_rank1.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_045000_rank3.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_045000_rank6.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_045000_rank7.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...eckpoints/model_045000.pt: 69%|██████▉ | 1.62GB / 2.35GB  ...nts/optim_045000_rank2.pt: 96%|█████████▌| 2.39GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.7GB / 22.2GB, 854MB/s New Data Upload : 85%|████████▌ | 5.45GB / 6.38GB, 234MB/s  ...nts/optim_045000_rank5.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_045000_rank4.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_045000_rank0.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_045000_rank1.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_045000_rank3.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_045000_rank6.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_045000_rank7.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...eckpoints/model_045000.pt: 70%|███████ | 1.66GB / 2.35GB  ...nts/optim_045000_rank2.pt: 96%|█████████▋| 2.39GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.7GB / 22.2GB, 837MB/s New Data Upload : 86%|████████▌ | 5.50GB / 6.38GB, 236MB/s  ...nts/optim_045000_rank5.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_045000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_045000_rank0.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_045000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_045000_rank3.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_045000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_045000_rank7.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...eckpoints/model_045000.pt: 71%|███████▏ | 1.68GB / 2.35GB  ...nts/optim_045000_rank2.pt: 96%|█████████▋| 2.40GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.8GB / 22.2GB, 809MB/s New Data Upload : 87%|████████▋ | 5.53GB / 6.38GB, 233MB/s  ...nts/optim_045000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_045000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_045000_rank0.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_045000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_045000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_045000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_045000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...eckpoints/model_045000.pt: 72%|███████▏ | 1.70GB / 2.35GB  ...nts/optim_045000_rank2.pt: 97%|█████████▋| 2.40GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 20.9GB / 22.2GB, 792MB/s New Data Upload : 87%|████████▋ | 5.56GB / 6.38GB, 232MB/s  ...nts/optim_045000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_045000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_045000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_045000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_045000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_045000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_045000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...eckpoints/model_045000.pt: 73%|███████▎ | 1.72GB / 2.35GB  ...nts/optim_045000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 20.9GB / 22.2GB, 777MB/s New Data Upload : 88%|████████▊ | 5.60GB / 6.38GB, 231MB/s  ...nts/optim_045000_rank5.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_045000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_045000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_045000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_045000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_045000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_045000_rank7.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...eckpoints/model_045000.pt: 74%|███████▍ | 1.74GB / 2.35GB  ...nts/optim_045000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 21.0GB / 22.2GB, 745MB/s New Data Upload : 88%|████████▊ | 5.63GB / 6.38GB, 228MB/s  ...nts/optim_045000_rank5.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_045000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_045000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_045000_rank1.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_045000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_045000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_045000_rank7.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...eckpoints/model_045000.pt: 75%|███████▌ | 1.77GB / 2.35GB  ...nts/optim_045000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▍| 21.0GB / 22.2GB, 714MB/s New Data Upload : 89%|████████▊ | 5.66GB / 6.38GB, 227MB/s  ...nts/optim_045000_rank5.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_045000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_045000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_045000_rank1.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_045000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_045000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_045000_rank7.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...eckpoints/model_045000.pt: 76%|███████▌ | 1.79GB / 2.35GB  ...nts/optim_045000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▍| 21.1GB / 22.2GB, 687MB/s New Data Upload : 89%|████████▉ | 5.68GB / 6.38GB, 225MB/s  ...nts/optim_045000_rank5.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_045000_rank4.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_045000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_045000_rank1.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_045000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank6.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_045000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...eckpoints/model_045000.pt: 77%|███████▋ | 1.81GB / 2.35GB  ...nts/optim_045000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.1GB / 22.2GB, 669MB/s New Data Upload : 90%|████████▉ | 5.72GB / 6.38GB, 224MB/s  ...nts/optim_045000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_045000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank0.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_045000_rank1.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_045000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...eckpoints/model_045000.pt: 78%|███████▊ | 1.83GB / 2.35GB  ...nts/optim_045000_rank2.pt: 98%|█████████▊| 2.42GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.2GB / 22.2GB, 649MB/s New Data Upload : 90%|████████▉ | 5.74GB / 6.38GB, 221MB/s  ...nts/optim_045000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_045000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank1.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_045000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...eckpoints/model_045000.pt: 79%|███████▉ | 1.85GB / 2.35GB  ...nts/optim_045000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.2GB / 22.2GB, 628MB/s New Data Upload : 90%|█████████ | 5.77GB / 6.38GB, 219MB/s  ...nts/optim_045000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_045000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_045000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...eckpoints/model_045000.pt: 80%|███████▉ | 1.87GB / 2.35GB  ...nts/optim_045000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 606MB/s New Data Upload : 91%|█████████ | 5.80GB / 6.38GB, 216MB/s  ...nts/optim_045000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_045000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_045000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank7.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...eckpoints/model_045000.pt: 81%|████████ | 1.89GB / 2.35GB  ...nts/optim_045000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 589MB/s New Data Upload : 91%|█████████▏| 5.83GB / 6.38GB, 215MB/s  ...nts/optim_045000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_045000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...eckpoints/model_045000.pt: 81%|████████▏ | 1.91GB / 2.35GB  ...nts/optim_045000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.4GB / 22.2GB, 574MB/s New Data Upload : 92%|█████████▏| 5.85GB / 6.38GB, 214MB/s  ...nts/optim_045000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_045000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...eckpoints/model_045000.pt: 82%|████████▏ | 1.93GB / 2.35GB  ...nts/optim_045000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▋| 21.4GB / 22.2GB, 554MB/s New Data Upload : 92%|█████████▏| 5.88GB / 6.38GB, 212MB/s  ...nts/optim_045000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_045000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank6.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...eckpoints/model_045000.pt: 83%|████████▎ | 1.94GB / 2.35GB  ...nts/optim_045000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 533MB/s New Data Upload : 92%|█████████▏| 5.90GB / 6.38GB, 210MB/s  ...nts/optim_045000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank4.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_045000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank6.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...eckpoints/model_045000.pt: 83%|████████▎ | 1.96GB / 2.35GB  ...nts/optim_045000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 510MB/s New Data Upload : 93%|█████████▎| 5.92GB / 6.38GB, 207MB/s  ...nts/optim_045000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank4.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank6.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...eckpoints/model_045000.pt: 84%|████████▍ | 1.97GB / 2.35GB  ...nts/optim_045000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 482MB/s New Data Upload : 93%|█████████▎| 5.94GB / 6.38GB, 205MB/s  ...nts/optim_045000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank7.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_045000.pt: 85%|████████▍ | 1.99GB / 2.35GB  ...nts/optim_045000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 462MB/s New Data Upload : 93%|█████████▎| 5.96GB / 6.38GB, 202MB/s  ...nts/optim_045000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank7.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_045000.pt: 85%|████████▌ | 2.01GB / 2.35GB  ...nts/optim_045000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 445MB/s New Data Upload : 94%|█████████▎| 5.98GB / 6.38GB, 200MB/s  ...nts/optim_045000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank7.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_045000.pt: 86%|████████▌ | 2.02GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 437MB/s New Data Upload : 94%|█████████▍| 6.00GB / 6.38GB, 198MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_045000.pt: 87%|████████▋ | 2.03GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 424MB/s New Data Upload : 94%|█████████▍| 6.02GB / 6.38GB, 195MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_045000.pt: 87%|████████▋ | 2.05GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.7GB / 22.2GB, 411MB/s New Data Upload : 95%|█████████▍| 6.04GB / 6.38GB, 192MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_045000.pt: 88%|████████▊ | 2.07GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 400MB/s New Data Upload : 95%|█████████▍| 6.06GB / 6.38GB, 190MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_045000.pt: 89%|████████▊ | 2.08GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 386MB/s New Data Upload : 95%|█████████▌| 6.07GB / 6.38GB, 186MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank1.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_045000.pt: 89%|████████▉ | 2.09GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 376MB/s New Data Upload : 95%|█████████▌| 6.09GB / 6.38GB, 183MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_045000.pt: 90%|████████▉ | 2.11GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 364MB/s New Data Upload : 96%|█████████▌| 6.11GB / 6.38GB, 180MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_045000.pt: 91%|█████████ | 2.13GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 353MB/s New Data Upload : 96%|█████████▌| 6.13GB / 6.38GB, 177MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_045000.pt: 91%|█████████ | 2.13GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 337MB/s New Data Upload : 96%|█████████▌| 6.13GB / 6.38GB, 172MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_045000.pt: 91%|█████████ | 2.14GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 331MB/s New Data Upload : 96%|█████████▋| 6.15GB / 6.38GB, 170MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_045000.pt: 92%|█████████▏| 2.15GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 317MB/s New Data Upload : 97%|█████████▋| 6.16GB / 6.38GB, 165MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_045000.pt: 92%|█████████▏| 2.16GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 303MB/s New Data Upload : 97%|█████████▋| 6.17GB / 6.38GB, 161MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_045000.pt: 92%|█████████▏| 2.17GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 291MB/s New Data Upload : 97%|█████████▋| 6.18GB / 6.38GB, 156MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 93%|█████████▎| 2.18GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 281MB/s New Data Upload : 97%|█████████▋| 6.19GB / 6.38GB, 152MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 93%|█████████▎| 2.19GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 268MB/s New Data Upload : 97%|█████████▋| 6.20GB / 6.38GB, 148MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 94%|█████████▎| 2.20GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 257MB/s New Data Upload : 97%|█████████▋| 6.21GB / 6.38GB, 144MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 94%|█████████▍| 2.21GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 249MB/s New Data Upload : 97%|█████████▋| 6.22GB / 6.38GB, 140MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 94%|█████████▍| 2.21GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 240MB/s New Data Upload : 98%|█████████▊| 6.23GB / 6.38GB, 137MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 95%|█████████▍| 2.23GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 232MB/s New Data Upload : 98%|█████████▊| 6.24GB / 6.38GB, 134MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 95%|█████████▌| 2.24GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 223MB/s New Data Upload : 98%|█████████▊| 6.25GB / 6.38GB, 130MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 96%|█████████▌| 2.24GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 216MB/s New Data Upload : 98%|█████████▊| 6.26GB / 6.38GB, 126MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 96%|█████████▌| 2.25GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 209MB/s New Data Upload : 98%|█████████▊| 6.27GB / 6.38GB, 122MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 96%|█████████▌| 2.26GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 203MB/s New Data Upload : 98%|█████████▊| 6.28GB / 6.38GB, 119MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 195MB/s New Data Upload : 98%|█████████▊| 6.29GB / 6.38GB, 115MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 187MB/s New Data Upload : 99%|█████████▊| 6.29GB / 6.38GB, 112MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 183MB/s New Data Upload : 99%|█████████▊| 6.30GB / 6.38GB, 109MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 97%|█████████▋| 2.29GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 174MB/s New Data Upload : 99%|█████████▉| 6.30GB / 6.38GB, 104MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 97%|█████████▋| 2.29GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 169MB/s New Data Upload : 99%|█████████▉| 6.31GB / 6.38GB, 101MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 98%|█████████▊| 2.29GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 163MB/s New Data Upload : 99%|█████████▉| 6.32GB / 6.38GB, 97.1MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 153MB/s New Data Upload : 99%|█████████▉| 6.32GB / 6.38GB, 92.5MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 148MB/s New Data Upload : 99%|█████████▉| 6.32GB / 6.38GB, 88.7MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 141MB/s New Data Upload : 99%|█████████▉| 6.33GB / 6.38GB, 85.8MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_045000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 134MB/s New Data Upload : 99%|█████████▉| 6.33GB / 6.38GB, 81.6MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 129MB/s New Data Upload : 99%|█████████▉| 6.33GB / 6.38GB, 78.6MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 122MB/s New Data Upload : 99%|█████████▉| 6.34GB / 6.38GB, 75.8MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 117MB/s New Data Upload : 99%|█████████▉| 6.34GB / 6.38GB, 72.8MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 111MB/s New Data Upload : 99%|█████████▉| 6.34GB / 6.38GB, 70.3MB/s  ...nts/optim_045000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 106MB/s New Data Upload : 99%|█████████▉| 6.35GB / 6.38GB, 67.3MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 104MB/s New Data Upload : 99%|█████████▉| 6.35GB / 6.38GB, 65.3MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 97.6MB/s New Data Upload : 100%|█████████▉| 6.36GB / 6.38GB, 62.6MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 99%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 94.0MB/s New Data Upload : 100%|█████████▉| 6.36GB / 6.38GB, 60.7MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 90.1MB/s New Data Upload : 100%|█████████▉| 6.36GB / 6.38GB, 58.2MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 86.2MB/s New Data Upload : 100%|█████████▉| 6.37GB / 6.38GB, 55.7MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 79.9MB/s New Data Upload : 100%|█████████▉| 6.37GB / 6.38GB, 53.0MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 72.2MB/s New Data Upload : 100%|█████████▉| 6.37GB / 6.38GB, 48.1MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 66.9MB/s New Data Upload : 100%|█████████▉| 6.37GB / 6.38GB, 44.6MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 64.5MB/s New Data Upload : 100%|█████████▉| 6.37GB / 6.38GB, 42.6MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 60.5MB/s New Data Upload : 100%|█████████▉| 6.37GB / 6.38GB, 40.6MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 57.5MB/s New Data Upload : 100%|█████████▉| 6.38GB / 6.38GB, 36.8MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 44.2MB/s New Data Upload : 100%|█████████▉| 6.38GB / 6.38GB, 28.3MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (1 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 38.6MB/s New Data Upload : 100%|█████████▉| 6.38GB / 6.38GB, 23.9MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (1 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 33.5MB/s New Data Upload : 100%|█████████▉| 6.38GB / 6.38GB, 21.2MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (1 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 28.6MB/s New Data Upload : 100%|█████████▉| 6.38GB / 6.38GB, 17.4MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (1 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 24.7MB/s New Data Upload : 100%|█████████▉| 6.38GB / 6.38GB, 13.5MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (1 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 21.2MB/s New Data Upload : 100%|█████████▉| 6.38GB / 6.38GB, 10.4MB/s  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (9 / 9) : 100%|██████████| 22.2GB / 22.2GB, 12.8MB/s New Data Upload : 100%|██████████| 6.38GB / 6.38GB, 4.45MB/s  ...nts/optim_045000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_045000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_045000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (9 / 9) : 100%|██████████| 22.2GB / 22.2GB, 8.04MB/s New Data Upload : 100%|██████████| 6.38GB / 6.38GB, 1.60MB/s ...nts/optim_045000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_045000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_045000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_045000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_045000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_045000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_045000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB ...eckpoints/model_045000.pt: 100%|██████████| 2.35GB / 2.35GB ...nts/optim_045000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB [15:47:53] Step 45000 uploaded [15:57:53] Uploading step 50000... Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...eckpoints/model_050000.pt: 0%| | 590kB / 2.35GB  ...eckpoints/model_050000.pt: 0%| | 590kB / 2.35GB  Processing Files (0 / 1) : 0%| | 590kB / 22.2GB, 590kB/s New Data Upload : 0%| | 590kB / 604MB, 590kB/s  ...eckpoints/model_050000.pt: 0%| | 590kB / 2.35GB  ...eckpoints/model_050000.pt: 0%| | 1.18MB / 2.35GB  Processing Files (0 / 1) : 0%| | 1.18MB / 22.2GB, 842kB/s New Data Upload : 0%| | 1.18MB / 604MB, 842kB/s  ...eckpoints/model_050000.pt: 0%| | 5.31MB / 2.35GB  Processing Files (0 / 1) : 0%| | 5.31MB / 22.2GB, 3.32MB/s New Data Upload : 1%| | 5.31MB / 604MB, 3.32MB/s  ...eckpoints/model_050000.pt: 0%| | 7.67MB / 2.35GB  Processing Files (0 / 1) : 0%| | 7.67MB / 22.2GB, 4.26MB/s New Data Upload : 1%|▏ | 7.67MB / 604MB, 4.26MB/s  ...eckpoints/model_050000.pt: 1%| | 11.8MB / 2.35GB  Processing Files (0 / 1) : 0%| | 11.8MB / 22.2GB, 5.90MB/s New Data Upload : 2%|▏ | 11.8MB / 604MB, 5.90MB/s  ...eckpoints/model_050000.pt: 1%| | 20.6MB / 2.35GB  Processing Files (0 / 1) : 0%| | 20.6MB / 22.2GB, 9.38MB/s New Data Upload : 3%|▎ | 20.6MB / 604MB, 9.38MB/s  ...eckpoints/model_050000.pt: 1%|▏ | 29.5MB / 2.35GB  Processing Files (0 / 1) : 0%| | 29.5MB / 22.2GB, 12.3MB/s New Data Upload : 5%|▍ | 29.5MB / 604MB, 12.3MB/s  ...eckpoints/model_050000.pt: 2%|▏ | 38.9MB / 2.35GB  Processing Files (0 / 1) : 0%| | 38.9MB / 22.2GB, 15.0MB/s New Data Upload : 6%|▋ | 38.9MB / 604MB, 15.0MB/s  ...eckpoints/model_050000.pt: 2%|▏ | 44.8MB / 2.35GB  Processing Files (0 / 1) : 0%| | 44.8MB / 22.2GB, 16.0MB/s New Data Upload : 7%|▋ | 44.8MB / 604MB, 16.0MB/s  ...eckpoints/model_050000.pt: 2%|▏ | 50.7MB / 2.35GB  Processing Files (0 / 1) : 0%| | 50.7MB / 22.2GB, 16.9MB/s New Data Upload : 8%|▊ | 50.7MB / 604MB, 16.9MB/s  ...eckpoints/model_050000.pt: 2%|▏ | 55.4MB / 2.35GB  Processing Files (0 / 1) : 0%| | 55.4MB / 22.2GB, 17.3MB/s New Data Upload : 9%|▉ | 55.4MB / 604MB, 17.3MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 63.7MB / 2.35GB  Processing Files (0 / 1) : 0%| | 63.7MB / 22.2GB, 18.7MB/s New Data Upload : 11%|█ | 63.7MB / 604MB, 18.7MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 66.6MB / 2.35GB  Processing Files (0 / 1) : 0%| | 66.6MB / 22.2GB, 18.5MB/s New Data Upload : 11%|█ | 66.6MB / 604MB, 18.5MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 66.6MB / 2.35GB  ...eckpoints/model_050000.pt: 3%|▎ | 66.6MB / 2.35GB  ...eckpoints/model_050000.pt: 3%|▎ | 66.6MB / 2.35GB  ...eckpoints/model_050000.pt: 3%|▎ | 66.6MB / 2.35GB  ...nts/optim_050000_rank0.pt: 0%| | 4.16MB / 2.49GB  ...nts/optim_050000_rank2.pt: 0%| | 247kB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 247kB / 2.49GB  ...nts/optim_050000_rank4.pt: 0%| | 1.17MB / 2.49GB  ...nts/optim_050000_rank5.pt: 0%| | 247kB / 2.49GB  ...nts/optim_050000_rank6.pt: 0%| | 247kB / 2.49GB  ...nts/optim_050000_rank7.pt: 0%| | 247kB / 2.49GB  ...eckpoints/model_050000.pt: 3%|▎ | 67.0MB / 2.35GB  ...nts/optim_050000_rank0.pt: 0%| | 4.16MB / 2.49GB  ...nts/optim_050000_rank2.pt: 0%| | 247kB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 247kB / 2.49GB  ...nts/optim_050000_rank4.pt: 0%| | 1.17MB / 2.49GB  ...nts/optim_050000_rank5.pt: 0%| | 247kB / 2.49GB  ...nts/optim_050000_rank6.pt: 0%| | 247kB / 2.49GB  ...nts/optim_050000_rank7.pt: 0%| | 247kB / 2.49GB  Processing Files (0 / 8) : 0%| | 73.6MB / 22.2GB, 16.0MB/s New Data Upload : 12%|█▏ | 72.3MB / 604MB, 15.7MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 67.0MB / 2.35GB  ...nts/optim_050000_rank0.pt: 0%| | 7.68MB / 2.49GB  ...nts/optim_050000_rank2.pt: 0%| | 412kB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 412kB / 2.49GB  ...nts/optim_050000_rank4.pt: 0%| | 1.17MB / 2.49GB  ...nts/optim_050000_rank5.pt: 0%| | 412kB / 2.49GB  ...nts/optim_050000_rank6.pt: 0%| | 412kB / 2.49GB  ...nts/optim_050000_rank7.pt: 0%| | 412kB / 2.49GB  Processing Files (0 / 8) : 0%| | 78.0MB / 22.2GB, 16.2MB/s New Data Upload : 10%|█ | 75.8MB / 738MB, 15.8MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 67.0MB / 2.35GB  ...nts/optim_050000_rank0.pt: 0%| | 12.4MB / 2.49GB  ...nts/optim_050000_rank2.pt: 0%| | 632kB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 632kB / 2.49GB  ...nts/optim_050000_rank4.pt: 0%| | 1.17MB / 2.49GB  ...nts/optim_050000_rank5.pt: 0%| | 632kB / 2.49GB  ...nts/optim_050000_rank6.pt: 0%| | 632kB / 2.49GB  ...nts/optim_050000_rank7.pt: 0%| | 632kB / 2.49GB  Processing Files (0 / 8) : 0%| | 83.7MB / 22.2GB, 16.7MB/s New Data Upload : 11%|█ | 80.5MB / 738MB, 16.1MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 67.0MB / 2.35GB  ...nts/optim_050000_rank0.pt: 1%| | 20.1MB / 2.49GB  ...nts/optim_050000_rank2.pt: 0%| | 1.07MB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 1.07MB / 2.49GB  ...nts/optim_050000_rank4.pt: 0%| | 2.93MB / 2.49GB  ...nts/optim_050000_rank5.pt: 0%| | 1.07MB / 2.49GB  ...nts/optim_050000_rank6.pt: 0%| | 1.07MB / 2.49GB  ...nts/optim_050000_rank7.pt: 0%| | 1.07MB / 2.49GB  Processing Files (0 / 8) : 0%| | 95.4MB / 22.2GB, 18.4MB/s New Data Upload : 12%|█▏ | 89.9MB / 738MB, 17.3MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 67.0MB / 2.35GB  ...nts/optim_050000_rank0.pt: 1%| | 29.0MB / 2.49GB  ...nts/optim_050000_rank2.pt: 0%| | 1.57MB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 1.57MB / 2.49GB  ...nts/optim_050000_rank4.pt: 0%| | 4.71MB / 2.49GB  ...nts/optim_050000_rank5.pt: 0%| | 1.57MB / 2.49GB  ...nts/optim_050000_rank6.pt: 0%| | 1.57MB / 2.49GB  ...nts/optim_050000_rank7.pt: 0%| | 2.15MB / 2.49GB  Processing Files (0 / 8) : 0%| | 109MB / 22.2GB, 20.2MB/s New Data Upload : 13%|█▎ | 101MB / 805MB, 18.7MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 67.0MB / 2.35GB  ...nts/optim_050000_rank0.pt: 2%|▏ | 38.7MB / 2.49GB  ...nts/optim_050000_rank2.pt: 0%| | 2.34MB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 2.34MB / 2.49GB  ...nts/optim_050000_rank4.pt: 0%| | 11.8MB / 2.49GB  ...nts/optim_050000_rank5.pt: 0%| | 2.34MB / 2.49GB  ...nts/optim_050000_rank6.pt: 0%| | 2.34MB / 2.49GB  ...nts/optim_050000_rank7.pt: 0%| | 3.51MB / 2.49GB  Processing Files (0 / 8) : 1%| | 130MB / 22.2GB, 23.3MB/s New Data Upload : 15%|█▍ | 118MB / 805MB, 21.1MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 67.0MB / 2.35GB  ...nts/optim_050000_rank0.pt: 2%|▏ | 48.5MB / 2.49GB  ...nts/optim_050000_rank2.pt: 0%| | 3.75MB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 3.16MB / 2.49GB  ...nts/optim_050000_rank4.pt: 1%| | 20.0MB / 2.49GB  ...nts/optim_050000_rank5.pt: 0%| | 3.16MB / 2.49GB  ...nts/optim_050000_rank6.pt: 0%| | 3.16MB / 2.49GB  ...nts/optim_050000_rank7.pt: 0%| | 6.68MB / 2.49GB  Processing Files (0 / 8) : 1%| | 156MB / 22.2GB, 26.8MB/s New Data Upload : 17%|█▋ | 139MB / 805MB, 23.9MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 67.0MB / 2.35GB  ...nts/optim_050000_rank0.pt: 2%|▏ | 53.4MB / 2.49GB  ...nts/optim_050000_rank2.pt: 0%| | 4.13MB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 3.55MB / 2.49GB  ...nts/optim_050000_rank4.pt: 1%| | 23.6MB / 2.49GB  ...nts/optim_050000_rank5.pt: 0%| | 3.55MB / 2.49GB  ...nts/optim_050000_rank6.pt: 0%| | 3.55MB / 2.49GB  ...nts/optim_050000_rank7.pt: 0%| | 9.41MB / 2.49GB  Processing Files (0 / 8) : 1%| | 168MB / 22.2GB, 28.0MB/s New Data Upload : 17%|█▋ | 149MB / 872MB, 24.9MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 67.0MB / 2.35GB  ...nts/optim_050000_rank0.pt: 3%|▎ | 62.8MB / 2.49GB  ...nts/optim_050000_rank2.pt: 0%| | 5.52MB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 4.34MB / 2.49GB  ...nts/optim_050000_rank4.pt: 1%|▏ | 31.9MB / 2.49GB  ...nts/optim_050000_rank5.pt: 0%| | 4.34MB / 2.49GB  ...nts/optim_050000_rank6.pt: 0%| | 4.34MB / 2.49GB  ...nts/optim_050000_rank7.pt: 1%| | 16.1MB / 2.49GB  Processing Files (0 / 8) : 1%| | 196MB / 22.2GB, 31.7MB/s New Data Upload : 20%|█▉ | 173MB / 872MB, 27.8MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 67.0MB / 2.35GB  ...nts/optim_050000_rank0.pt: 3%|▎ | 68.0MB / 2.49GB  ...nts/optim_050000_rank2.pt: 0%| | 9.59MB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 4.89MB / 2.49GB  ...nts/optim_050000_rank4.pt: 2%|▏ | 39.2MB / 2.49GB  ...nts/optim_050000_rank5.pt: 0%| | 5.94MB / 2.49GB  ...nts/optim_050000_rank6.pt: 0%| | 4.89MB / 2.49GB  ...nts/optim_050000_rank7.pt: 1%| | 25.4MB / 2.49GB  Processing Files (0 / 8) : 1%| | 225MB / 22.2GB, 35.1MB/s New Data Upload : 23%|██▎ | 197MB / 872MB, 30.8MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 67.0MB / 2.35GB  ...nts/optim_050000_rank0.pt: 3%|▎ | 68.3MB / 2.49GB  ...nts/optim_050000_rank2.pt: 0%| | 11.6MB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 5.11MB / 2.49GB  ...nts/optim_050000_rank4.pt: 2%|▏ | 44.0MB / 2.49GB  ...nts/optim_050000_rank5.pt: 0%| | 6.29MB / 2.49GB  ...nts/optim_050000_rank6.pt: 0%| | 5.11MB / 2.49GB  ...nts/optim_050000_rank7.pt: 1%|▏ | 31.5MB / 2.49GB  Processing Files (0 / 8) : 1%| | 239MB / 22.2GB, 36.2MB/s New Data Upload : 22%|██▏ | 210MB / 939MB, 31.8MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 67.0MB / 2.35GB  ...nts/optim_050000_rank0.pt: 3%|▎ | 70.0MB / 2.49GB  ...nts/optim_050000_rank2.pt: 0%| | 12.0MB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 5.53MB / 2.49GB  ...nts/optim_050000_rank4.pt: 2%|▏ | 51.8MB / 2.49GB  ...nts/optim_050000_rank5.pt: 0%| | 8.60MB / 2.49GB  ...nts/optim_050000_rank6.pt: 0%| | 6.31MB / 2.49GB  ...nts/optim_050000_rank7.pt: 2%|▏ | 38.4MB / 2.49GB  Processing Files (0 / 8) : 1%| | 260MB / 22.2GB, 38.2MB/s New Data Upload : 24%|██▍ | 227MB / 939MB, 33.3MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 67.0MB / 2.35GB  ...nts/optim_050000_rank0.pt: 3%|▎ | 70.3MB / 2.49GB  ...nts/optim_050000_rank2.pt: 1%| | 13.9MB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 5.66MB / 2.49GB  ...nts/optim_050000_rank4.pt: 2%|▏ | 54.9MB / 2.49GB  ...nts/optim_050000_rank5.pt: 0%| | 11.7MB / 2.49GB  ...nts/optim_050000_rank6.pt: 0%| | 6.55MB / 2.49GB  ...nts/optim_050000_rank7.pt: 2%|▏ | 40.3MB / 2.49GB  Processing Files (0 / 8) : 1%| | 270MB / 22.2GB, 38.6MB/s New Data Upload : 25%|██▌ | 236MB / 939MB, 33.7MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 67.0MB / 2.35GB  ...nts/optim_050000_rank0.pt: 3%|▎ | 70.7MB / 2.49GB  ...nts/optim_050000_rank2.pt: 1%| | 15.9MB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 5.90MB / 2.49GB  ...nts/optim_050000_rank4.pt: 2%|▏ | 59.7MB / 2.49GB  ...nts/optim_050000_rank5.pt: 1%| | 15.0MB / 2.49GB  ...nts/optim_050000_rank6.pt: 0%| | 7.53MB / 2.49GB  ...nts/optim_050000_rank7.pt: 2%|▏ | 45.2MB / 2.49GB  Processing Files (0 / 8) : 1%|▏ | 287MB / 22.2GB, 39.8MB/s New Data Upload : 27%|██▋ | 251MB / 939MB, 34.8MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 67.0MB / 2.35GB  ...nts/optim_050000_rank0.pt: 3%|▎ | 71.2MB / 2.49GB  ...nts/optim_050000_rank2.pt: 1%| | 16.0MB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 7.50MB / 2.49GB  ...nts/optim_050000_rank4.pt: 3%|▎ | 63.6MB / 2.49GB  ...nts/optim_050000_rank5.pt: 1%| | 24.0MB / 2.49GB  ...nts/optim_050000_rank6.pt: 0%| | 10.3MB / 2.49GB  ...nts/optim_050000_rank7.pt: 2%|▏ | 48.3MB / 2.49GB  Processing Files (0 / 8) : 1%|▏ | 308MB / 22.2GB, 41.6MB/s New Data Upload : 27%|██▋ | 268MB / 1.01GB, 36.2MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 67.0MB / 2.35GB  ...nts/optim_050000_rank0.pt: 3%|▎ | 71.6MB / 2.49GB  ...nts/optim_050000_rank2.pt: 1%| | 18.0MB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 7.99MB / 2.49GB  ...nts/optim_050000_rank4.pt: 3%|▎ | 67.9MB / 2.49GB  ...nts/optim_050000_rank5.pt: 1%| | 27.8MB / 2.49GB  ...nts/optim_050000_rank6.pt: 1%| | 14.1MB / 2.49GB  ...nts/optim_050000_rank7.pt: 2%|▏ | 50.8MB / 2.49GB  Processing Files (0 / 8) : 1%|▏ | 325MB / 22.2GB, 42.8MB/s New Data Upload : 28%|██▊ | 283MB / 1.01GB, 37.3MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 67.0MB / 2.35GB  ...nts/optim_050000_rank0.pt: 3%|▎ | 72.1MB / 2.49GB  ...nts/optim_050000_rank2.pt: 1%| | 19.8MB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 8.87MB / 2.49GB  ...nts/optim_050000_rank4.pt: 3%|▎ | 68.7MB / 2.49GB  ...nts/optim_050000_rank5.pt: 1%|▏ | 35.5MB / 2.49GB  ...nts/optim_050000_rank6.pt: 1%| | 14.4MB / 2.49GB  ...nts/optim_050000_rank7.pt: 2%|▏ | 55.0MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 341MB / 22.2GB, 43.8MB/s New Data Upload : 30%|██▉ | 298MB / 1.01GB, 38.2MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 67.6MB / 2.35GB  ...nts/optim_050000_rank0.pt: 3%|▎ | 72.4MB / 2.49GB  ...nts/optim_050000_rank2.pt: 1%| | 21.5MB / 2.49GB  ...nts/optim_050000_rank3.pt: 0%| | 11.0MB / 2.49GB  ...nts/optim_050000_rank4.pt: 3%|▎ | 69.1MB / 2.49GB  ...nts/optim_050000_rank5.pt: 2%|▏ | 43.2MB / 2.49GB  ...nts/optim_050000_rank6.pt: 1%| | 17.0MB / 2.49GB  ...nts/optim_050000_rank7.pt: 2%|▏ | 57.9MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 360MB / 22.2GB, 45.0MB/s New Data Upload : 28%|██▊ | 315MB / 1.14GB, 39.4MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 68.2MB / 2.35GB  ...nts/optim_050000_rank0.pt: 3%|▎ | 72.8MB / 2.49GB  ...nts/optim_050000_rank2.pt: 1%| | 23.3MB / 2.49GB  ...nts/optim_050000_rank3.pt: 1%| | 13.1MB / 2.49GB  ...nts/optim_050000_rank4.pt: 3%|▎ | 70.0MB / 2.49GB  ...nts/optim_050000_rank5.pt: 2%|▏ | 48.0MB / 2.49GB  ...nts/optim_050000_rank6.pt: 1%| | 19.6MB / 2.49GB  ...nts/optim_050000_rank7.pt: 2%|▏ | 61.5MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 377MB / 22.2GB, 45.9MB/s New Data Upload : 29%|██▉ | 331MB / 1.14GB, 40.3MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 72.3MB / 2.35GB  ...nts/optim_050000_rank0.pt: 3%|▎ | 73.4MB / 2.49GB  ...nts/optim_050000_rank2.pt: 1%| | 25.2MB / 2.49GB  ...nts/optim_050000_rank3.pt: 1%| | 17.7MB / 2.49GB  ...nts/optim_050000_rank4.pt: 3%|▎ | 71.4MB / 2.49GB  ...nts/optim_050000_rank5.pt: 2%|▏ | 57.0MB / 2.49GB  ...nts/optim_050000_rank6.pt: 1%| | 24.6MB / 2.49GB  ...nts/optim_050000_rank7.pt: 3%|▎ | 64.5MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 406MB / 22.2GB, 48.3MB/s New Data Upload : 30%|██▉ | 358MB / 1.21GB, 42.6MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 72.3MB / 2.35GB  ...nts/optim_050000_rank0.pt: 3%|▎ | 79.0MB / 2.49GB  ...nts/optim_050000_rank2.pt: 1%|▏ | 32.0MB / 2.49GB  ...nts/optim_050000_rank3.pt: 1%| | 28.9MB / 2.49GB  ...nts/optim_050000_rank4.pt: 3%|▎ | 75.3MB / 2.49GB  ...nts/optim_050000_rank5.pt: 3%|▎ | 62.1MB / 2.49GB  ...nts/optim_050000_rank6.pt: 1%|▏ | 31.6MB / 2.49GB  ...nts/optim_050000_rank7.pt: 3%|▎ | 73.2MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 454MB / 22.2GB, 52.8MB/s New Data Upload : 31%|███▏ | 378MB / 1.21GB, 44.0MB/s  ...eckpoints/model_050000.pt: 3%|▎ | 79.3MB / 2.35GB  ...nts/optim_050000_rank0.pt: 3%|▎ | 86.2MB / 2.49GB  ...nts/optim_050000_rank2.pt: 2%|▏ | 38.4MB / 2.49GB  ...nts/optim_050000_rank3.pt: 2%|▏ | 42.0MB / 2.49GB  ...nts/optim_050000_rank4.pt: 3%|▎ | 79.6MB / 2.49GB  ...nts/optim_050000_rank5.pt: 3%|▎ | 76.2MB / 2.49GB  ...nts/optim_050000_rank6.pt: 2%|▏ | 40.5MB / 2.49GB  ...nts/optim_050000_rank7.pt: 3%|▎ | 83.0MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 525MB / 22.2GB, 59.7MB/s New Data Upload : 34%|███▍ | 415MB / 1.21GB, 47.1MB/s  ...eckpoints/model_050000.pt: 4%|▎ | 87.4MB / 2.35GB  ...nts/optim_050000_rank0.pt: 4%|▍ | 95.3MB / 2.49GB  ...nts/optim_050000_rank2.pt: 2%|▏ | 48.6MB / 2.49GB  ...nts/optim_050000_rank3.pt: 2%|▏ | 54.9MB / 2.49GB  ...nts/optim_050000_rank4.pt: 4%|▎ | 87.6MB / 2.49GB  ...nts/optim_050000_rank5.pt: 4%|▎ | 92.8MB / 2.49GB  ...nts/optim_050000_rank6.pt: 2%|▏ | 52.6MB / 2.49GB  ...nts/optim_050000_rank7.pt: 4%|▎ | 92.6MB / 2.49GB  Processing Files (0 / 8) : 3%|▎ | 612MB / 22.2GB, 68.0MB/s New Data Upload : 38%|███▊ | 455MB / 1.21GB, 50.6MB/s  ...eckpoints/model_050000.pt: 4%|▍ | 95.6MB / 2.35GB  ...nts/optim_050000_rank0.pt: 4%|▍ | 103MB / 2.49GB  ...nts/optim_050000_rank2.pt: 2%|▏ | 58.3MB / 2.49GB  ...nts/optim_050000_rank3.pt: 3%|▎ | 69.5MB / 2.49GB  ...nts/optim_050000_rank4.pt: 4%|▍ | 96.7MB / 2.49GB  ...nts/optim_050000_rank5.pt: 4%|▍ | 102MB / 2.49GB  ...nts/optim_050000_rank6.pt: 3%|▎ | 63.5MB / 2.49GB  ...nts/optim_050000_rank7.pt: 4%|▍ | 101MB / 2.49GB  Processing Files (0 / 8) : 3%|▎ | 689MB / 22.2GB, 74.9MB/s New Data Upload : 37%|███▋ | 491MB / 1.34GB, 53.4MB/s  ...eckpoints/model_050000.pt: 4%|▍ | 103MB / 2.35GB  ...nts/optim_050000_rank0.pt: 5%|▍ | 114MB / 2.49GB  ...nts/optim_050000_rank2.pt: 3%|▎ | 70.3MB / 2.49GB  ...nts/optim_050000_rank3.pt: 3%|▎ | 83.9MB / 2.49GB  ...nts/optim_050000_rank4.pt: 4%|▍ | 105MB / 2.49GB  ...nts/optim_050000_rank5.pt: 5%|▍ | 112MB / 2.49GB  ...nts/optim_050000_rank6.pt: 3%|▎ | 75.5MB / 2.49GB  ...nts/optim_050000_rank7.pt: 4%|▍ | 111MB / 2.49GB  Processing Files (0 / 8) : 3%|▎ | 774MB / 22.2GB, 82.4MB/s New Data Upload : 39%|███▉ | 520MB / 1.34GB, 55.4MB/s  ...eckpoints/model_050000.pt: 5%|▍ | 108MB / 2.35GB  ...nts/optim_050000_rank0.pt: 5%|▌ | 133MB / 2.49GB  ...nts/optim_050000_rank2.pt: 4%|▎ | 89.0MB / 2.49GB  ...nts/optim_050000_rank3.pt: 4%|▍ | 107MB / 2.49GB  ...nts/optim_050000_rank4.pt: 5%|▌ | 125MB / 2.49GB  ...nts/optim_050000_rank5.pt: 5%|▌ | 131MB / 2.49GB  ...nts/optim_050000_rank6.pt: 4%|▍ | 94.1MB / 2.49GB  ...nts/optim_050000_rank7.pt: 5%|▌ | 131MB / 2.49GB  Processing Files (0 / 8) : 4%|▍ | 918MB / 22.2GB, 95.6MB/s New Data Upload : 42%|████▏ | 558MB / 1.34GB, 58.2MB/s  ...eckpoints/model_050000.pt: 5%|▍ | 117MB / 2.35GB  ...nts/optim_050000_rank0.pt: 6%|▌ | 150MB / 2.49GB  ...nts/optim_050000_rank2.pt: 4%|▍ | 108MB / 2.49GB  ...nts/optim_050000_rank3.pt: 5%|▌ | 126MB / 2.49GB  ...nts/optim_050000_rank4.pt: 6%|▌ | 138MB / 2.49GB  ...nts/optim_050000_rank5.pt: 6%|▌ | 148MB / 2.49GB  ...nts/optim_050000_rank6.pt: 5%|▍ | 112MB / 2.49GB  ...nts/optim_050000_rank7.pt: 6%|▌ | 148MB / 2.49GB  Processing Files (0 / 8) : 5%|▍ | 1.05GB / 22.2GB, 107MB/s New Data Upload : 42%|████▏ | 594MB / 1.41GB, 60.6MB/s  ...eckpoints/model_050000.pt: 5%|▌ | 122MB / 2.35GB  ...nts/optim_050000_rank0.pt: 6%|▋ | 160MB / 2.49GB  ...nts/optim_050000_rank2.pt: 5%|▍ | 120MB / 2.49GB  ...nts/optim_050000_rank3.pt: 6%|▌ | 141MB / 2.49GB  ...nts/optim_050000_rank4.pt: 6%|▌ | 150MB / 2.49GB  ...nts/optim_050000_rank5.pt: 6%|▋ | 158MB / 2.49GB  ...nts/optim_050000_rank6.pt: 5%|▌ | 125MB / 2.49GB  ...nts/optim_050000_rank7.pt: 6%|▋ | 158MB / 2.49GB  Processing Files (0 / 8) : 5%|▌ | 1.14GB / 22.2GB, 114MB/s New Data Upload : 44%|████▍ | 623MB / 1.41GB, 62.3MB/s  ...eckpoints/model_050000.pt: 5%|▌ | 128MB / 2.35GB  ...nts/optim_050000_rank0.pt: 7%|▋ | 182MB / 2.49GB  ...nts/optim_050000_rank2.pt: 6%|▌ | 142MB / 2.49GB  ...nts/optim_050000_rank3.pt: 7%|▋ | 164MB / 2.49GB  ...nts/optim_050000_rank4.pt: 7%|▋ | 172MB / 2.49GB  ...nts/optim_050000_rank5.pt: 7%|▋ | 179MB / 2.49GB  ...nts/optim_050000_rank6.pt: 6%|▌ | 148MB / 2.49GB  ...nts/optim_050000_rank7.pt: 7%|▋ | 179MB / 2.49GB  Processing Files (0 / 8) : 6%|▌ | 1.29GB / 22.2GB, 127MB/s New Data Upload : 45%|████▍ | 660MB / 1.48GB, 64.7MB/s  ...eckpoints/model_050000.pt: 6%|▌ | 133MB / 2.35GB  ...nts/optim_050000_rank0.pt: 8%|▊ | 196MB / 2.49GB  ...nts/optim_050000_rank2.pt: 6%|▋ | 158MB / 2.49GB  ...nts/optim_050000_rank3.pt: 7%|▋ | 181MB / 2.49GB  ...nts/optim_050000_rank4.pt: 7%|▋ | 184MB / 2.49GB  ...nts/optim_050000_rank5.pt: 8%|▊ | 193MB / 2.49GB  ...nts/optim_050000_rank6.pt: 7%|▋ | 163MB / 2.49GB  ...nts/optim_050000_rank7.pt: 8%|▊ | 193MB / 2.49GB  Processing Files (0 / 8) : 6%|▋ | 1.40GB / 22.2GB, 137MB/s New Data Upload : 47%|████▋ | 686MB / 1.48GB, 67.3MB/s  ...eckpoints/model_050000.pt: 6%|▌ | 133MB / 2.35GB  ...nts/optim_050000_rank0.pt: 9%|▉ | 221MB / 2.49GB  ...nts/optim_050000_rank2.pt: 7%|▋ | 185MB / 2.49GB  ...nts/optim_050000_rank3.pt: 8%|▊ | 209MB / 2.49GB  ...nts/optim_050000_rank4.pt: 9%|▊ | 216MB / 2.49GB  ...nts/optim_050000_rank5.pt: 9%|▉ | 219MB / 2.49GB  ...nts/optim_050000_rank6.pt: 8%|▊ | 190MB / 2.49GB  ...nts/optim_050000_rank7.pt: 9%|▉ | 219MB / 2.49GB  Processing Files (0 / 8) : 7%|▋ | 1.59GB / 22.2GB, 156MB/s New Data Upload : 47%|████▋ | 725MB / 1.54GB, 71.1MB/s  ...eckpoints/model_050000.pt: 6%|▌ | 135MB / 2.35GB  ...nts/optim_050000_rank0.pt: 10%|▉ | 242MB / 2.49GB  ...nts/optim_050000_rank2.pt: 8%|▊ | 208MB / 2.49GB  ...nts/optim_050000_rank3.pt: 9%|▉ | 232MB / 2.49GB  ...nts/optim_050000_rank4.pt: 9%|▉ | 234MB / 2.49GB  ...nts/optim_050000_rank5.pt: 10%|▉ | 239MB / 2.49GB  ...nts/optim_050000_rank6.pt: 9%|▊ | 212MB / 2.49GB  ...nts/optim_050000_rank7.pt: 10%|▉ | 239MB / 2.49GB  Processing Files (0 / 8) : 8%|▊ | 1.74GB / 22.2GB, 171MB/s New Data Upload : 45%|████▌ | 759MB / 1.68GB, 74.4MB/s  ...eckpoints/model_050000.pt: 6%|▌ | 135MB / 2.35GB  ...nts/optim_050000_rank0.pt: 11%|█ | 262MB / 2.49GB  ...nts/optim_050000_rank2.pt: 9%|▉ | 228MB / 2.49GB  ...nts/optim_050000_rank3.pt: 10%|█ | 254MB / 2.49GB  ...nts/optim_050000_rank4.pt: 10%|█ | 253MB / 2.49GB  ...nts/optim_050000_rank5.pt: 10%|█ | 260MB / 2.49GB  ...nts/optim_050000_rank6.pt: 9%|▉ | 235MB / 2.49GB  ...nts/optim_050000_rank7.pt: 10%|█ | 260MB / 2.49GB  Processing Files (0 / 8) : 8%|▊ | 1.89GB / 22.2GB, 185MB/s New Data Upload : 47%|████▋ | 786MB / 1.68GB, 77.1MB/s  ...eckpoints/model_050000.pt: 6%|▌ | 136MB / 2.35GB  ...nts/optim_050000_rank0.pt: 11%|█ | 275MB / 2.49GB  ...nts/optim_050000_rank2.pt: 10%|▉ | 243MB / 2.49GB  ...nts/optim_050000_rank3.pt: 11%|█ | 269MB / 2.49GB  ...nts/optim_050000_rank4.pt: 11%|█ | 268MB / 2.49GB  ...nts/optim_050000_rank5.pt: 11%|█ | 273MB / 2.49GB  ...nts/optim_050000_rank6.pt: 10%|█ | 249MB / 2.49GB  ...nts/optim_050000_rank7.pt: 11%|█ | 273MB / 2.49GB  Processing Files (0 / 8) : 9%|▉ | 1.99GB / 22.2GB, 195MB/s New Data Upload : 48%|████▊ | 811MB / 1.68GB, 79.4MB/s  ...eckpoints/model_050000.pt: 6%|▌ | 140MB / 2.35GB  ...nts/optim_050000_rank0.pt: 12%|█▏ | 288MB / 2.49GB  ...nts/optim_050000_rank2.pt: 10%|█ | 257MB / 2.49GB  ...nts/optim_050000_rank3.pt: 11%|█▏ | 285MB / 2.49GB  ...nts/optim_050000_rank4.pt: 11%|█▏ | 285MB / 2.49GB  ...nts/optim_050000_rank5.pt: 11%|█▏ | 285MB / 2.49GB  ...nts/optim_050000_rank6.pt: 11%|█ | 265MB / 2.49GB  ...nts/optim_050000_rank7.pt: 11%|█▏ | 286MB / 2.49GB  Processing Files (0 / 8) : 9%|▉ | 2.09GB / 22.2GB, 205MB/s New Data Upload : 48%|████▊ | 840MB / 1.74GB, 82.3MB/s  ...eckpoints/model_050000.pt: 6%|▌ | 146MB / 2.35GB  ...nts/optim_050000_rank0.pt: 12%|█▏ | 300MB / 2.49GB  ...nts/optim_050000_rank2.pt: 11%|█ | 269MB / 2.49GB  ...nts/optim_050000_rank3.pt: 12%|█▏ | 298MB / 2.49GB  ...nts/optim_050000_rank4.pt: 12%|█▏ | 298MB / 2.49GB  ...nts/optim_050000_rank5.pt: 12%|█▏ | 297MB / 2.49GB  ...nts/optim_050000_rank6.pt: 11%|█ | 276MB / 2.49GB  ...nts/optim_050000_rank7.pt: 12%|█▏ | 297MB / 2.49GB  Processing Files (0 / 8) : 10%|▉ | 2.18GB / 22.2GB, 214MB/s New Data Upload : 48%|████▊ | 862MB / 1.81GB, 84.3MB/s  ...eckpoints/model_050000.pt: 6%|▋ | 150MB / 2.35GB  ...nts/optim_050000_rank0.pt: 13%|█▎ | 321MB / 2.49GB  ...nts/optim_050000_rank2.pt: 12%|█▏ | 291MB / 2.49GB  ...nts/optim_050000_rank3.pt: 13%|█▎ | 318MB / 2.49GB  ...nts/optim_050000_rank4.pt: 13%|█▎ | 318MB / 2.49GB  ...nts/optim_050000_rank5.pt: 13%|█▎ | 316MB / 2.49GB  ...nts/optim_050000_rank6.pt: 12%|█▏ | 297MB / 2.49GB  ...nts/optim_050000_rank7.pt: 13%|█▎ | 317MB / 2.49GB  Processing Files (0 / 8) : 10%|█ | 2.33GB / 22.2GB, 228MB/s New Data Upload : 49%|████▉ | 893MB / 1.81GB, 87.0MB/s  ...eckpoints/model_050000.pt: 7%|▋ | 153MB / 2.35GB  ...nts/optim_050000_rank0.pt: 14%|█▎ | 339MB / 2.49GB  ...nts/optim_050000_rank2.pt: 12%|█▏ | 308MB / 2.49GB  ...nts/optim_050000_rank3.pt: 13%|█▎ | 334MB / 2.49GB  ...nts/optim_050000_rank4.pt: 13%|█▎ | 334MB / 2.49GB  ...nts/optim_050000_rank5.pt: 13%|█▎ | 332MB / 2.49GB  ...nts/optim_050000_rank6.pt: 13%|█▎ | 315MB / 2.49GB  ...nts/optim_050000_rank7.pt: 13%|█▎ | 333MB / 2.49GB  Processing Files (0 / 8) : 11%|█ | 2.45GB / 22.2GB, 239MB/s New Data Upload : 51%|█████ | 919MB / 1.81GB, 89.3MB/s  ...eckpoints/model_050000.pt: 7%|▋ | 158MB / 2.35GB  ...nts/optim_050000_rank0.pt: 14%|█▍ | 349MB / 2.49GB  ...nts/optim_050000_rank2.pt: 13%|█▎ | 318MB / 2.49GB  ...nts/optim_050000_rank3.pt: 14%|█▍ | 343MB / 2.49GB  ...nts/optim_050000_rank4.pt: 14%|█▍ | 343MB / 2.49GB  ...nts/optim_050000_rank5.pt: 14%|█▎ | 341MB / 2.49GB  ...nts/optim_050000_rank6.pt: 13%|█▎ | 326MB / 2.49GB  ...nts/optim_050000_rank7.pt: 14%|█▎ | 342MB / 2.49GB  Processing Files (0 / 8) : 11%|█▏ | 2.52GB / 22.2GB, 246MB/s New Data Upload : 50%|████▉ | 936MB / 1.88GB, 90.6MB/s  ...eckpoints/model_050000.pt: 7%|▋ | 161MB / 2.35GB  ...nts/optim_050000_rank0.pt: 15%|█▍ | 368MB / 2.49GB  ...nts/optim_050000_rank2.pt: 14%|█▎ | 339MB / 2.49GB  ...nts/optim_050000_rank3.pt: 15%|█▍ | 363MB / 2.49GB  ...nts/optim_050000_rank4.pt: 15%|█▍ | 362MB / 2.49GB  ...nts/optim_050000_rank5.pt: 15%|█▍ | 361MB / 2.49GB  ...nts/optim_050000_rank6.pt: 14%|█▍ | 347MB / 2.49GB  ...nts/optim_050000_rank7.pt: 15%|█▍ | 361MB / 2.49GB  Processing Files (0 / 8) : 12%|█▏ | 2.66GB / 22.2GB, 259MB/s New Data Upload : 50%|████▉ | 966MB / 1.94GB, 92.7MB/s  ...eckpoints/model_050000.pt: 7%|▋ | 168MB / 2.35GB  ...nts/optim_050000_rank0.pt: 16%|█▌ | 386MB / 2.49GB  ...nts/optim_050000_rank2.pt: 14%|█▍ | 354MB / 2.49GB  ...nts/optim_050000_rank3.pt: 15%|█▌ | 378MB / 2.49GB  ...nts/optim_050000_rank4.pt: 15%|█▌ | 378MB / 2.49GB  ...nts/optim_050000_rank5.pt: 15%|█▌ | 375MB / 2.49GB  ...nts/optim_050000_rank6.pt: 15%|█▍ | 362MB / 2.49GB  ...nts/optim_050000_rank7.pt: 15%|█▌ | 375MB / 2.49GB  Processing Files (0 / 8) : 12%|█▏ | 2.78GB / 22.2GB, 269MB/s New Data Upload : 51%|█████ | 993MB / 1.94GB, 94.5MB/s  ...eckpoints/model_050000.pt: 8%|▊ | 177MB / 2.35GB  ...nts/optim_050000_rank0.pt: 16%|█▌ | 403MB / 2.49GB  ...nts/optim_050000_rank2.pt: 15%|█▌ | 374MB / 2.49GB  ...nts/optim_050000_rank3.pt: 16%|█▌ | 396MB / 2.49GB  ...nts/optim_050000_rank4.pt: 16%|█▌ | 396MB / 2.49GB  ...nts/optim_050000_rank5.pt: 16%|█▌ | 394MB / 2.49GB  ...nts/optim_050000_rank6.pt: 15%|█▌ | 383MB / 2.49GB  ...nts/optim_050000_rank7.pt: 16%|█▌ | 394MB / 2.49GB  Processing Files (0 / 8) : 13%|█▎ | 2.92GB / 22.2GB, 282MB/s New Data Upload : 51%|█████ | 1.03GB / 2.01GB, 96.9MB/s  ...eckpoints/model_050000.pt: 8%|▊ | 184MB / 2.35GB  ...nts/optim_050000_rank0.pt: 17%|█▋ | 424MB / 2.49GB  ...nts/optim_050000_rank2.pt: 16%|█▌ | 397MB / 2.49GB  ...nts/optim_050000_rank3.pt: 17%|█▋ | 419MB / 2.49GB  ...nts/optim_050000_rank4.pt: 17%|█▋ | 418MB / 2.49GB  ...nts/optim_050000_rank5.pt: 17%|█▋ | 417MB / 2.49GB  ...nts/optim_050000_rank6.pt: 16%|█▋ | 407MB / 2.49GB  ...nts/optim_050000_rank7.pt: 17%|█▋ | 417MB / 2.49GB  Processing Files (0 / 8) : 14%|█▍ | 3.08GB / 22.2GB, 298MB/s New Data Upload : 51%|█████ | 1.06GB / 2.08GB, 99.9MB/s  ...eckpoints/model_050000.pt: 8%|▊ | 189MB / 2.35GB  ...nts/optim_050000_rank0.pt: 18%|█▊ | 451MB / 2.49GB  ...nts/optim_050000_rank2.pt: 17%|█▋ | 424MB / 2.49GB  ...nts/optim_050000_rank3.pt: 18%|█▊ | 448MB / 2.49GB  ...nts/optim_050000_rank4.pt: 18%|█▊ | 445MB / 2.49GB  ...nts/optim_050000_rank5.pt: 18%|█▊ | 444MB / 2.49GB  ...nts/optim_050000_rank6.pt: 17%|█▋ | 435MB / 2.49GB  ...nts/optim_050000_rank7.pt: 18%|█▊ | 444MB / 2.49GB  Processing Files (0 / 8) : 15%|█▍ | 3.28GB / 22.2GB, 317MB/s New Data Upload : 53%|█████▎ | 1.10GB / 2.08GB, 103MB/s  ...eckpoints/model_050000.pt: 8%|▊ | 195MB / 2.35GB  ...nts/optim_050000_rank0.pt: 19%|█▉ | 479MB / 2.49GB  ...nts/optim_050000_rank2.pt: 18%|█▊ | 453MB / 2.49GB  ...nts/optim_050000_rank3.pt: 19%|█▉ | 474MB / 2.49GB  ...nts/optim_050000_rank4.pt: 19%|█▉ | 472MB / 2.49GB  ...nts/optim_050000_rank5.pt: 19%|█▉ | 471MB / 2.49GB  ...nts/optim_050000_rank6.pt: 19%|█▊ | 462MB / 2.49GB  ...nts/optim_050000_rank7.pt: 19%|█▉ | 470MB / 2.49GB  Processing Files (0 / 8) : 16%|█▌ | 3.48GB / 22.2GB, 335MB/s New Data Upload : 53%|█████▎ | 1.14GB / 2.15GB, 107MB/s  ...eckpoints/model_050000.pt: 8%|▊ | 199MB / 2.35GB  ...nts/optim_050000_rank0.pt: 20%|██ | 500MB / 2.49GB  ...nts/optim_050000_rank2.pt: 19%|█▉ | 476MB / 2.49GB  ...nts/optim_050000_rank3.pt: 20%|█▉ | 496MB / 2.49GB  ...nts/optim_050000_rank4.pt: 20%|█▉ | 495MB / 2.49GB  ...nts/optim_050000_rank5.pt: 20%|█▉ | 494MB / 2.49GB  ...nts/optim_050000_rank6.pt: 20%|█▉ | 487MB / 2.49GB  ...nts/optim_050000_rank7.pt: 20%|█▉ | 494MB / 2.49GB  Processing Files (0 / 8) : 16%|█▋ | 3.64GB / 22.2GB, 351MB/s New Data Upload : 50%|█████ | 1.18GB / 2.35GB, 109MB/s  ...eckpoints/model_050000.pt: 9%|▉ | 206MB / 2.35GB  ...nts/optim_050000_rank0.pt: 21%|██▏ | 531MB / 2.49GB  ...nts/optim_050000_rank2.pt: 21%|██ | 511MB / 2.49GB  ...nts/optim_050000_rank3.pt: 21%|██▏ | 533MB / 2.49GB  ...nts/optim_050000_rank4.pt: 21%|██ | 528MB / 2.49GB  ...nts/optim_050000_rank5.pt: 21%|██ | 527MB / 2.49GB  ...nts/optim_050000_rank6.pt: 21%|██ | 522MB / 2.49GB  ...nts/optim_050000_rank7.pt: 21%|██ | 527MB / 2.49GB  Processing Files (0 / 8) : 17%|█▋ | 3.88GB / 22.2GB, 374MB/s New Data Upload : 51%|█████ | 1.23GB / 2.41GB, 114MB/s  ...eckpoints/model_050000.pt: 9%|▉ | 210MB / 2.35GB  ...nts/optim_050000_rank0.pt: 23%|██▎ | 560MB / 2.49GB  ...nts/optim_050000_rank2.pt: 22%|██▏ | 540MB / 2.49GB  ...nts/optim_050000_rank3.pt: 23%|██▎ | 566MB / 2.49GB  ...nts/optim_050000_rank4.pt: 22%|██▏ | 558MB / 2.49GB  ...nts/optim_050000_rank5.pt: 22%|██▏ | 557MB / 2.49GB  ...nts/optim_050000_rank6.pt: 22%|██▏ | 552MB / 2.49GB  ...nts/optim_050000_rank7.pt: 22%|██▏ | 557MB / 2.49GB  Processing Files (0 / 8) : 18%|█▊ | 4.10GB / 22.2GB, 395MB/s New Data Upload : 51%|█████▏ | 1.28GB / 2.48GB, 119MB/s  ...eckpoints/model_050000.pt: 9%|▉ | 213MB / 2.35GB  ...nts/optim_050000_rank0.pt: 24%|██▎ | 585MB / 2.49GB  ...nts/optim_050000_rank2.pt: 23%|██▎ | 567MB / 2.49GB  ...nts/optim_050000_rank3.pt: 24%|██▍ | 593MB / 2.49GB  ...nts/optim_050000_rank4.pt: 23%|██▎ | 582MB / 2.49GB  ...nts/optim_050000_rank5.pt: 23%|██▎ | 581MB / 2.49GB  ...nts/optim_050000_rank6.pt: 23%|██▎ | 579MB / 2.49GB  ...nts/optim_050000_rank7.pt: 23%|██▎ | 582MB / 2.49GB  Processing Files (0 / 8) : 19%|█▉ | 4.28GB / 22.2GB, 413MB/s New Data Upload : 53%|█████▎ | 1.32GB / 2.48GB, 123MB/s  ...eckpoints/model_050000.pt: 9%|▉ | 216MB / 2.35GB  ...nts/optim_050000_rank0.pt: 25%|██▍ | 613MB / 2.49GB  ...nts/optim_050000_rank2.pt: 24%|██▍ | 598MB / 2.49GB  ...nts/optim_050000_rank3.pt: 25%|██▌ | 624MB / 2.49GB  ...nts/optim_050000_rank4.pt: 25%|██▍ | 613MB / 2.49GB  ...nts/optim_050000_rank5.pt: 25%|██▍ | 612MB / 2.49GB  ...nts/optim_050000_rank6.pt: 25%|██▍ | 610MB / 2.49GB  ...nts/optim_050000_rank7.pt: 25%|██▍ | 613MB / 2.49GB  Processing Files (0 / 8) : 20%|██ | 4.50GB / 22.2GB, 435MB/s New Data Upload : 55%|█████▍ | 1.36GB / 2.48GB, 127MB/s  ...eckpoints/model_050000.pt: 9%|▉ | 218MB / 2.35GB  ...nts/optim_050000_rank0.pt: 26%|██▌ | 643MB / 2.49GB  ...nts/optim_050000_rank2.pt: 25%|██▌ | 628MB / 2.49GB  ...nts/optim_050000_rank3.pt: 26%|██▋ | 654MB / 2.49GB  ...nts/optim_050000_rank4.pt: 26%|██▌ | 644MB / 2.49GB  ...nts/optim_050000_rank5.pt: 26%|██▌ | 643MB / 2.49GB  ...nts/optim_050000_rank6.pt: 26%|██▌ | 642MB / 2.49GB  ...nts/optim_050000_rank7.pt: 26%|██▌ | 644MB / 2.49GB  Processing Files (0 / 8) : 21%|██ | 4.72GB / 22.2GB, 456MB/s New Data Upload : 55%|█████▌ | 1.40GB / 2.55GB, 131MB/s  ...eckpoints/model_050000.pt: 10%|▉ | 225MB / 2.35GB  ...nts/optim_050000_rank0.pt: 27%|██▋ | 666MB / 2.49GB  ...nts/optim_050000_rank2.pt: 26%|██▋ | 655MB / 2.49GB  ...nts/optim_050000_rank3.pt: 27%|██▋ | 681MB / 2.49GB  ...nts/optim_050000_rank4.pt: 27%|██▋ | 668MB / 2.49GB  ...nts/optim_050000_rank5.pt: 27%|██▋ | 667MB / 2.49GB  ...nts/optim_050000_rank6.pt: 27%|██▋ | 669MB / 2.49GB  ...nts/optim_050000_rank7.pt: 27%|██▋ | 669MB / 2.49GB  Processing Files (0 / 8) : 22%|██▏ | 4.90GB / 22.2GB, 473MB/s New Data Upload : 57%|█████▋ | 1.45GB / 2.55GB, 135MB/s  ...eckpoints/model_050000.pt: 10%|▉ | 225MB / 2.35GB  ...nts/optim_050000_rank0.pt: 28%|██▊ | 698MB / 2.49GB  ...nts/optim_050000_rank2.pt: 28%|██▊ | 687MB / 2.49GB  ...nts/optim_050000_rank3.pt: 29%|██▊ | 713MB / 2.49GB  ...nts/optim_050000_rank4.pt: 28%|██▊ | 701MB / 2.49GB  ...nts/optim_050000_rank5.pt: 28%|██▊ | 699MB / 2.49GB  ...nts/optim_050000_rank6.pt: 28%|██▊ | 700MB / 2.49GB  ...nts/optim_050000_rank7.pt: 28%|██▊ | 701MB / 2.49GB  Processing Files (0 / 8) : 23%|██▎ | 5.12GB / 22.2GB, 495MB/s New Data Upload : 57%|█████▋ | 1.49GB / 2.62GB, 138MB/s  ...eckpoints/model_050000.pt: 10%|▉ | 229MB / 2.35GB  ...nts/optim_050000_rank0.pt: 29%|██▉ | 730MB / 2.49GB  ...nts/optim_050000_rank2.pt: 29%|██▉ | 719MB / 2.49GB  ...nts/optim_050000_rank3.pt: 30%|██▉ | 744MB / 2.49GB  ...nts/optim_050000_rank4.pt: 29%|██▉ | 732MB / 2.49GB  ...nts/optim_050000_rank5.pt: 29%|██▉ | 730MB / 2.49GB  ...nts/optim_050000_rank6.pt: 29%|██▉ | 733MB / 2.49GB  ...nts/optim_050000_rank7.pt: 29%|██▉ | 732MB / 2.49GB  Processing Files (0 / 8) : 24%|██▍ | 5.35GB / 22.2GB, 516MB/s New Data Upload : 58%|█████▊ | 1.53GB / 2.62GB, 142MB/s  ...eckpoints/model_050000.pt: 10%|▉ | 232MB / 2.35GB  ...nts/optim_050000_rank0.pt: 31%|███ | 767MB / 2.49GB  ...nts/optim_050000_rank2.pt: 30%|███ | 757MB / 2.49GB  ...nts/optim_050000_rank3.pt: 32%|███▏ | 785MB / 2.49GB  ...nts/optim_050000_rank4.pt: 31%|███ | 770MB / 2.49GB  ...nts/optim_050000_rank5.pt: 31%|███ | 767MB / 2.49GB  ...nts/optim_050000_rank6.pt: 31%|███ | 770MB / 2.49GB  ...nts/optim_050000_rank7.pt: 31%|███ | 769MB / 2.49GB  Processing Files (0 / 8) : 25%|██▌ | 5.62GB / 22.2GB, 541MB/s New Data Upload : 59%|█████▉ | 1.58GB / 2.68GB, 146MB/s  ...eckpoints/model_050000.pt: 10%|█ | 235MB / 2.35GB  ...nts/optim_050000_rank0.pt: 32%|███▏ | 800MB / 2.49GB  ...nts/optim_050000_rank2.pt: 32%|███▏ | 787MB / 2.49GB  ...nts/optim_050000_rank3.pt: 33%|███▎ | 814MB / 2.49GB  ...nts/optim_050000_rank4.pt: 32%|███▏ | 801MB / 2.49GB  ...nts/optim_050000_rank5.pt: 32%|███▏ | 797MB / 2.49GB  ...nts/optim_050000_rank6.pt: 32%|███▏ | 801MB / 2.49GB  ...nts/optim_050000_rank7.pt: 32%|███▏ | 799MB / 2.49GB  Processing Files (0 / 8) : 26%|██▌ | 5.83GB / 22.2GB, 561MB/s New Data Upload : 60%|██████ | 1.62GB / 2.68GB, 149MB/s  ...eckpoints/model_050000.pt: 10%|█ | 239MB / 2.35GB  ...nts/optim_050000_rank0.pt: 33%|███▎ | 829MB / 2.49GB  ...nts/optim_050000_rank2.pt: 33%|███▎ | 817MB / 2.49GB  ...nts/optim_050000_rank3.pt: 34%|███▍ | 843MB / 2.49GB  ...nts/optim_050000_rank4.pt: 33%|███▎ | 830MB / 2.49GB  ...nts/optim_050000_rank5.pt: 33%|███▎ | 825MB / 2.49GB  ...nts/optim_050000_rank6.pt: 33%|███▎ | 831MB / 2.49GB  ...nts/optim_050000_rank7.pt: 33%|███▎ | 828MB / 2.49GB  Processing Files (0 / 8) : 27%|██▋ | 6.04GB / 22.2GB, 580MB/s New Data Upload : 62%|██████▏ | 1.66GB / 2.68GB, 151MB/s  ...eckpoints/model_050000.pt: 10%|█ | 242MB / 2.35GB  ...nts/optim_050000_rank0.pt: 34%|███▍ | 842MB / 2.49GB  ...nts/optim_050000_rank2.pt: 33%|███▎ | 830MB / 2.49GB  ...nts/optim_050000_rank3.pt: 34%|███▍ | 854MB / 2.49GB  ...nts/optim_050000_rank4.pt: 34%|███▍ | 842MB / 2.49GB  ...nts/optim_050000_rank5.pt: 34%|███▎ | 838MB / 2.49GB  ...nts/optim_050000_rank6.pt: 34%|███▍ | 844MB / 2.49GB  ...nts/optim_050000_rank7.pt: 34%|███▍ | 840MB / 2.49GB  Processing Files (0 / 8) : 28%|██▊ | 6.13GB / 22.2GB, 586MB/s New Data Upload : 62%|██████▏ | 1.67GB / 2.68GB, 151MB/s  ...eckpoints/model_050000.pt: 10%|█ | 245MB / 2.35GB  ...nts/optim_050000_rank0.pt: 35%|███▌ | 871MB / 2.49GB  ...nts/optim_050000_rank2.pt: 35%|███▍ | 861MB / 2.49GB  ...nts/optim_050000_rank3.pt: 35%|███▌ | 881MB / 2.49GB  ...nts/optim_050000_rank4.pt: 35%|███▌ | 871MB / 2.49GB  ...nts/optim_050000_rank5.pt: 35%|███▍ | 867MB / 2.49GB  ...nts/optim_050000_rank6.pt: 35%|███▌ | 873MB / 2.49GB  ...nts/optim_050000_rank7.pt: 35%|███▍ | 869MB / 2.49GB  Processing Files (0 / 8) : 28%|██▊ | 6.34GB / 22.2GB, 605MB/s New Data Upload : 61%|██████ | 1.71GB / 2.82GB, 153MB/s  ...eckpoints/model_050000.pt: 11%|█ | 249MB / 2.35GB  ...nts/optim_050000_rank0.pt: 36%|███▌ | 892MB / 2.49GB  ...nts/optim_050000_rank2.pt: 36%|███▌ | 884MB / 2.49GB  ...nts/optim_050000_rank3.pt: 36%|███▌ | 900MB / 2.49GB  ...nts/optim_050000_rank4.pt: 36%|███▌ | 891MB / 2.49GB  ...nts/optim_050000_rank5.pt: 36%|███▌ | 888MB / 2.49GB  ...nts/optim_050000_rank6.pt: 36%|███▌ | 896MB / 2.49GB  ...nts/optim_050000_rank7.pt: 36%|███▌ | 890MB / 2.49GB  Processing Files (0 / 8) : 29%|██▉ | 6.49GB / 22.2GB, 617MB/s New Data Upload : 62%|██████▏ | 1.75GB / 2.82GB, 154MB/s  ...eckpoints/model_050000.pt: 11%|█ | 253MB / 2.35GB  ...nts/optim_050000_rank0.pt: 37%|███▋ | 916MB / 2.49GB  ...nts/optim_050000_rank2.pt: 37%|███▋ | 908MB / 2.49GB  ...nts/optim_050000_rank3.pt: 37%|███▋ | 922MB / 2.49GB  ...nts/optim_050000_rank4.pt: 37%|███▋ | 915MB / 2.49GB  ...nts/optim_050000_rank5.pt: 37%|███▋ | 914MB / 2.49GB  ...nts/optim_050000_rank6.pt: 37%|███▋ | 923MB / 2.49GB  ...nts/optim_050000_rank7.pt: 37%|███▋ | 915MB / 2.49GB  Processing Files (0 / 8) : 30%|██▉ | 6.67GB / 22.2GB, 631MB/s New Data Upload : 63%|██████▎ | 1.78GB / 2.82GB, 155MB/s  ...eckpoints/model_050000.pt: 11%|█ | 256MB / 2.35GB  ...nts/optim_050000_rank0.pt: 38%|███▊ | 943MB / 2.49GB  ...nts/optim_050000_rank2.pt: 38%|███▊ | 937MB / 2.49GB  ...nts/optim_050000_rank3.pt: 38%|███▊ | 947MB / 2.49GB  ...nts/optim_050000_rank4.pt: 38%|███▊ | 941MB / 2.49GB  ...nts/optim_050000_rank5.pt: 38%|███▊ | 941MB / 2.49GB  ...nts/optim_050000_rank6.pt: 38%|███▊ | 952MB / 2.49GB  ...nts/optim_050000_rank7.pt: 38%|███▊ | 943MB / 2.49GB  Processing Files (0 / 8) : 31%|███ | 6.86GB / 22.2GB, 649MB/s New Data Upload : 65%|██████▍ | 1.82GB / 2.82GB, 158MB/s  ...eckpoints/model_050000.pt: 11%|█ | 262MB / 2.35GB  ...nts/optim_050000_rank0.pt: 39%|███▉ | 967MB / 2.49GB  ...nts/optim_050000_rank2.pt: 39%|███▊ | 958MB / 2.49GB  ...nts/optim_050000_rank3.pt: 39%|███▉ | 969MB / 2.49GB  ...nts/optim_050000_rank4.pt: 39%|███▉ | 963MB / 2.49GB  ...nts/optim_050000_rank5.pt: 39%|███▉ | 967MB / 2.49GB  ...nts/optim_050000_rank6.pt: 39%|███▉ | 981MB / 2.49GB  ...nts/optim_050000_rank7.pt: 39%|███▉ | 968MB / 2.49GB  Processing Files (0 / 8) : 32%|███▏ | 7.03GB / 22.2GB, 664MB/s New Data Upload : 66%|██████▌ | 1.86GB / 2.82GB, 160MB/s  ...eckpoints/model_050000.pt: 11%|█ | 264MB / 2.35GB  ...nts/optim_050000_rank0.pt: 40%|███▉ | 993MB / 2.49GB  ...nts/optim_050000_rank2.pt: 40%|███▉ | 982MB / 2.49GB  ...nts/optim_050000_rank3.pt: 40%|███▉ | 992MB / 2.49GB  ...nts/optim_050000_rank4.pt: 40%|███▉ | 988MB / 2.49GB  ...nts/optim_050000_rank5.pt: 40%|████ | 994MB / 2.49GB  ...nts/optim_050000_rank6.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_050000_rank7.pt: 40%|████ | 995MB / 2.49GB  Processing Files (0 / 8) : 32%|███▏ | 7.22GB / 22.2GB, 681MB/s New Data Upload : 63%|██████▎ | 1.89GB / 3.02GB, 163MB/s  ...eckpoints/model_050000.pt: 11%|█▏ | 268MB / 2.35GB  ...nts/optim_050000_rank0.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_050000_rank2.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_050000_rank3.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_050000_rank4.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_050000_rank5.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_050000_rank6.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_050000_rank7.pt: 41%|████ | 1.02GB / 2.49GB  Processing Files (0 / 8) : 33%|███▎ | 7.40GB / 22.2GB, 697MB/s New Data Upload : 59%|█████▊ | 1.93GB / 3.29GB, 165MB/s  ...eckpoints/model_050000.pt: 12%|█▏ | 272MB / 2.35GB  ...nts/optim_050000_rank0.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_050000_rank2.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_050000_rank3.pt: 42%|████▏ | 1.03GB / 2.49GB  ...nts/optim_050000_rank4.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_050000_rank5.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_050000_rank6.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_050000_rank7.pt: 42%|████▏ | 1.04GB / 2.49GB  Processing Files (0 / 8) : 34%|███▍ | 7.56GB / 22.2GB, 711MB/s New Data Upload : 59%|█████▊ | 1.97GB / 3.35GB, 167MB/s  ...eckpoints/model_050000.pt: 12%|█▏ | 279MB / 2.35GB  ...nts/optim_050000_rank0.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_050000_rank2.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_050000_rank3.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_050000_rank4.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_050000_rank5.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_050000_rank6.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_050000_rank7.pt: 43%|████▎ | 1.07GB / 2.49GB  Processing Files (0 / 8) : 35%|███▍ | 7.78GB / 22.2GB, 731MB/s New Data Upload : 59%|█████▉ | 2.02GB / 3.42GB, 170MB/s  ...eckpoints/model_050000.pt: 12%|█▏ | 289MB / 2.35GB  ...nts/optim_050000_rank0.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_050000_rank2.pt: 44%|████▎ | 1.09GB / 2.49GB  ...nts/optim_050000_rank3.pt: 44%|████▎ | 1.09GB / 2.49GB  ...nts/optim_050000_rank4.pt: 44%|████▎ | 1.08GB / 2.49GB  ...nts/optim_050000_rank5.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_050000_rank6.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_050000_rank7.pt: 44%|████▍ | 1.10GB / 2.49GB  Processing Files (0 / 8) : 36%|███▌ | 7.98GB / 22.2GB, 749MB/s New Data Upload : 59%|█████▉ | 2.06GB / 3.49GB, 173MB/s  ...eckpoints/model_050000.pt: 13%|█▎ | 301MB / 2.35GB  ...nts/optim_050000_rank0.pt: 45%|████▌ | 1.13GB / 2.49GB  ...nts/optim_050000_rank2.pt: 45%|████▍ | 1.12GB / 2.49GB  ...nts/optim_050000_rank3.pt: 45%|████▍ | 1.12GB / 2.49GB  ...nts/optim_050000_rank4.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_050000_rank5.pt: 46%|████▌ | 1.13GB / 2.49GB  ...nts/optim_050000_rank6.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_050000_rank7.pt: 46%|████▌ | 1.13GB / 2.49GB  Processing Files (0 / 8) : 37%|███▋ | 8.21GB / 22.2GB, 770MB/s New Data Upload : 61%|██████ | 2.12GB / 3.49GB, 177MB/s  ...eckpoints/model_050000.pt: 13%|█▎ | 308MB / 2.35GB  ...nts/optim_050000_rank0.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_050000_rank2.pt: 46%|████▋ | 1.15GB / 2.49GB  ...nts/optim_050000_rank3.pt: 46%|████▌ | 1.15GB / 2.49GB  ...nts/optim_050000_rank4.pt: 46%|████▌ | 1.15GB / 2.49GB  ...nts/optim_050000_rank5.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_050000_rank6.pt: 49%|████▊ | 1.21GB / 2.49GB  ...nts/optim_050000_rank7.pt: 47%|████▋ | 1.16GB / 2.49GB  Processing Files (0 / 8) : 38%|███▊ | 8.45GB / 22.2GB, 792MB/s New Data Upload : 61%|██████ | 2.17GB / 3.55GB, 180MB/s  ...eckpoints/model_050000.pt: 13%|█▎ | 316MB / 2.35GB  ...nts/optim_050000_rank0.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_050000_rank2.pt: 48%|████▊ | 1.18GB / 2.49GB  ...nts/optim_050000_rank3.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_050000_rank4.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_050000_rank5.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_050000_rank6.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_050000_rank7.pt: 48%|████▊ | 1.19GB / 2.49GB  Processing Files (0 / 8) : 39%|███▉ | 8.65GB / 22.2GB, 808MB/s New Data Upload : 61%|██████ | 2.22GB / 3.62GB, 182MB/s  ...eckpoints/model_050000.pt: 14%|█▍ | 324MB / 2.35GB  ...nts/optim_050000_rank0.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_050000_rank2.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_050000_rank3.pt: 49%|████▊ | 1.21GB / 2.49GB  ...nts/optim_050000_rank4.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_050000_rank5.pt: 49%|████▉ | 1.23GB / 2.49GB  ...nts/optim_050000_rank6.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_050000_rank7.pt: 49%|████▉ | 1.22GB / 2.49GB  Processing Files (0 / 8) : 40%|███▉ | 8.88GB / 22.2GB, 826MB/s New Data Upload : 63%|██████▎ | 2.26GB / 3.62GB, 185MB/s  ...eckpoints/model_050000.pt: 14%|█▍ | 332MB / 2.35GB  ...nts/optim_050000_rank0.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_050000_rank2.pt: 50%|█████ | 1.24GB / 2.49GB  ...nts/optim_050000_rank3.pt: 50%|████▉ | 1.23GB / 2.49GB  ...nts/optim_050000_rank4.pt: 49%|████▉ | 1.23GB / 2.49GB  ...nts/optim_050000_rank5.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_050000_rank6.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_050000_rank7.pt: 50%|█████ | 1.24GB / 2.49GB  Processing Files (0 / 8) : 41%|████ | 9.06GB / 22.2GB, 837MB/s New Data Upload : 63%|██████▎ | 2.31GB / 3.69GB, 186MB/s  ...eckpoints/model_050000.pt: 14%|█▍ | 339MB / 2.35GB  ...nts/optim_050000_rank0.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_050000_rank2.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_050000_rank3.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_050000_rank4.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_050000_rank5.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_050000_rank6.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_050000_rank7.pt: 51%|█████ | 1.26GB / 2.49GB  Processing Files (0 / 8) : 41%|████▏ | 9.21GB / 22.2GB, 843MB/s New Data Upload : 64%|██████▎ | 2.35GB / 3.69GB, 185MB/s  ...eckpoints/model_050000.pt: 15%|█▍ | 345MB / 2.35GB  ...nts/optim_050000_rank0.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_050000_rank2.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_050000_rank3.pt: 52%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_050000_rank4.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_050000_rank5.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_050000_rank6.pt: 54%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_050000_rank7.pt: 52%|█████▏ | 1.29GB / 2.49GB  Processing Files (0 / 8) : 42%|████▏ | 9.41GB / 22.2GB, 855MB/s New Data Upload : 65%|██████▍ | 2.39GB / 3.69GB, 186MB/s  ...eckpoints/model_050000.pt: 15%|█▌ | 353MB / 2.35GB  ...nts/optim_050000_rank0.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_050000_rank2.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_050000_rank3.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_050000_rank4.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_050000_rank5.pt: 53%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_050000_rank6.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_050000_rank7.pt: 53%|█████▎ | 1.32GB / 2.49GB  Processing Files (0 / 8) : 43%|████▎ | 9.61GB / 22.2GB, 866MB/s New Data Upload : 65%|██████▌ | 2.44GB / 3.76GB, 188MB/s  ...eckpoints/model_050000.pt: 15%|█▌ | 359MB / 2.35GB  ...nts/optim_050000_rank0.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_050000_rank2.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_050000_rank3.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_050000_rank4.pt: 53%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_050000_rank5.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_050000_rank6.pt: 56%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_050000_rank7.pt: 54%|█████▍ | 1.34GB / 2.49GB  Processing Files (0 / 8) : 44%|████▍ | 9.81GB / 22.2GB, 871MB/s New Data Upload : 65%|██████▌ | 2.49GB / 3.82GB, 190MB/s  ...eckpoints/model_050000.pt: 16%|█▌ | 365MB / 2.35GB  ...nts/optim_050000_rank0.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_050000_rank2.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_050000_rank3.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_050000_rank4.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_050000_rank5.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_050000_rank6.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...nts/optim_050000_rank7.pt: 55%|█████▍ | 1.36GB / 2.49GB  Processing Files (0 / 8) : 45%|████▍ | 9.98GB / 22.2GB, 875MB/s New Data Upload : 65%|██████▌ | 2.54GB / 3.89GB, 190MB/s  ...eckpoints/model_050000.pt: 16%|█▌ | 372MB / 2.35GB  ...nts/optim_050000_rank0.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_050000_rank2.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_050000_rank3.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_050000_rank4.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_050000_rank5.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_050000_rank6.pt: 58%|█████▊ | 1.43GB / 2.49GB  ...nts/optim_050000_rank7.pt: 56%|█████▌ | 1.39GB / 2.49GB  Processing Files (0 / 8) : 46%|████▌ | 10.2GB / 22.2GB, 885MB/s New Data Upload : 65%|██████▌ | 2.59GB / 3.96GB, 192MB/s  ...eckpoints/model_050000.pt: 16%|█▌ | 377MB / 2.35GB  ...nts/optim_050000_rank0.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_050000_rank2.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_050000_rank3.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_050000_rank4.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...nts/optim_050000_rank5.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_050000_rank6.pt: 59%|█████▉ | 1.46GB / 2.49GB  ...nts/optim_050000_rank7.pt: 57%|█████▋ | 1.41GB / 2.49GB  Processing Files (0 / 8) : 47%|████▋ | 10.4GB / 22.2GB, 889MB/s New Data Upload : 64%|██████▍ | 2.64GB / 4.09GB, 194MB/s  ...eckpoints/model_050000.pt: 16%|█▌ | 381MB / 2.35GB  ...nts/optim_050000_rank0.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_050000_rank2.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_050000_rank3.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_050000_rank4.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_050000_rank5.pt: 59%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_050000_rank6.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 58%|█████▊ | 1.44GB / 2.49GB  Processing Files (0 / 8) : 48%|████▊ | 10.6GB / 22.2GB, 900MB/s New Data Upload : 65%|██████▍ | 2.69GB / 4.16GB, 196MB/s  ...eckpoints/model_050000.pt: 16%|█▋ | 385MB / 2.35GB  ...nts/optim_050000_rank0.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_050000_rank2.pt: 59%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_050000_rank3.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_050000_rank4.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_050000_rank5.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_050000_rank6.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_050000_rank7.pt: 59%|█████▊ | 1.46GB / 2.49GB  Processing Files (0 / 8) : 48%|████▊ | 10.7GB / 22.2GB, 896MB/s New Data Upload : 66%|██████▌ | 2.73GB / 4.16GB, 197MB/s  ...eckpoints/model_050000.pt: 17%|█▋ | 389MB / 2.35GB  ...nts/optim_050000_rank0.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_050000_rank2.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 61%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_050000_rank7.pt: 59%|█████▉ | 1.47GB / 2.49GB  Processing Files (0 / 8) : 49%|████▉ | 10.8GB / 22.2GB, 893MB/s New Data Upload : 64%|██████▍ | 2.77GB / 4.29GB, 197MB/s  ...eckpoints/model_050000.pt: 17%|█▋ | 392MB / 2.35GB  ...nts/optim_050000_rank0.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_050000_rank2.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_050000_rank3.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_050000_rank4.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_050000_rank5.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_050000_rank6.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_050000_rank7.pt: 60%|█████▉ | 1.49GB / 2.49GB  Processing Files (0 / 8) : 49%|████▉ | 11.0GB / 22.2GB, 890MB/s New Data Upload : 65%|██████▌ | 2.80GB / 4.29GB, 198MB/s  ...eckpoints/model_050000.pt: 17%|█▋ | 397MB / 2.35GB  ...nts/optim_050000_rank0.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_050000_rank2.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_050000_rank3.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_050000_rank4.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_050000_rank5.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_050000_rank6.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_050000_rank7.pt: 61%|██████ | 1.52GB / 2.49GB  Processing Files (0 / 8) : 50%|█████ | 11.2GB / 22.2GB, 901MB/s New Data Upload : 65%|██████▌ | 2.85GB / 4.36GB, 200MB/s  ...eckpoints/model_050000.pt: 17%|█▋ | 403MB / 2.35GB  ...nts/optim_050000_rank0.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_050000_rank2.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_050000_rank3.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_050000_rank4.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_050000_rank5.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_050000_rank6.pt: 64%|██████▍ | 1.58GB / 2.49GB  ...nts/optim_050000_rank7.pt: 62%|██████▏ | 1.53GB / 2.49GB  Processing Files (0 / 8) : 51%|█████ | 11.3GB / 22.2GB, 902MB/s New Data Upload : 66%|██████▌ | 2.88GB / 4.36GB, 200MB/s  ...nts/optim_050000_rank1.pt: 1%| | 15.9MB / 2.49GB  ...eckpoints/model_050000.pt: 17%|█▋ | 410MB / 2.35GB  ...nts/optim_050000_rank0.pt: 64%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_050000_rank2.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_050000_rank3.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_050000_rank4.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_050000_rank5.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_050000_rank6.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_050000_rank7.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_050000_rank1.pt: 1%| | 15.9MB / 2.49GB  Processing Files (0 / 9) : 52%|█████▏ | 11.5GB / 22.2GB, 910MB/s New Data Upload : 67%|██████▋ | 2.93GB / 4.36GB, 203MB/s  ...eckpoints/model_050000.pt: 18%|█▊ | 422MB / 2.35GB  ...nts/optim_050000_rank0.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_050000_rank2.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_050000_rank3.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_050000_rank4.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_050000_rank5.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_050000_rank6.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_050000_rank7.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_050000_rank1.pt: 6%|▌ | 142MB / 2.49GB  Processing Files (0 / 9) : 53%|█████▎ | 11.7GB / 22.2GB, 922MB/s New Data Upload : 68%|██████▊ | 2.97GB / 4.36GB, 204MB/s  ...eckpoints/model_050000.pt: 18%|█▊ | 432MB / 2.35GB  ...nts/optim_050000_rank0.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_050000_rank2.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_050000_rank3.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_050000_rank4.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_050000_rank5.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_050000_rank6.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_050000_rank7.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_050000_rank1.pt: 10%|█ | 254MB / 2.49GB  Processing Files (0 / 9) : 54%|█████▍ | 12.0GB / 22.2GB, 938MB/s New Data Upload : 65%|██████▌ | 3.01GB / 4.61GB, 205MB/s  ...eckpoints/model_050000.pt: 19%|█▉ | 443MB / 2.35GB  ...nts/optim_050000_rank0.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_050000_rank2.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_050000_rank3.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_050000_rank4.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_050000_rank5.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_050000_rank6.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_050000_rank7.pt: 65%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_050000_rank1.pt: 16%|█▌ | 397MB / 2.49GB  Processing Files (0 / 9) : 55%|█████▌ | 12.3GB / 22.2GB, 960MB/s New Data Upload : 65%|██████▌ | 3.05GB / 4.67GB, 207MB/s  ...eckpoints/model_050000.pt: 19%|█▉ | 450MB / 2.35GB  ...nts/optim_050000_rank0.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_050000_rank2.pt: 67%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_050000_rank3.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_050000_rank4.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_050000_rank5.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_050000_rank6.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_050000_rank7.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_050000_rank1.pt: 17%|█▋ | 429MB / 2.49GB  Processing Files (0 / 9) : 56%|█████▌ | 12.5GB / 22.2GB, 964MB/s New Data Upload : 66%|██████▋ | 3.09GB / 4.67GB, 209MB/s  ...eckpoints/model_050000.pt: 20%|█▉ | 469MB / 2.35GB  ...nts/optim_050000_rank0.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_050000_rank2.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_050000_rank3.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_050000_rank4.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_050000_rank5.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_050000_rank6.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_050000_rank7.pt: 66%|██████▌ | 1.65GB / 2.49GB  ...nts/optim_050000_rank1.pt: 25%|██▍ | 621MB / 2.49GB  Processing Files (0 / 9) : 58%|█████▊ | 12.8GB / 22.2GB, 986MB/s New Data Upload : 67%|██████▋ | 3.14GB / 4.67GB, 211MB/s  ...eckpoints/model_050000.pt: 20%|██ | 475MB / 2.35GB  ...nts/optim_050000_rank0.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_050000_rank2.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_050000_rank3.pt: 69%|██████▊ | 1.71GB / 2.49GB  ...nts/optim_050000_rank4.pt: 69%|██████▊ | 1.71GB / 2.49GB  ...nts/optim_050000_rank5.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_050000_rank6.pt: 69%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_050000_rank7.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_050000_rank1.pt: 31%|███ | 765MB / 2.49GB  Processing Files (0 / 9) : 59%|█████▉ | 13.2GB / 22.2GB, 1.00GB/s New Data Upload : 68%|██████▊ | 3.19GB / 4.67GB, 212MB/s  ...eckpoints/model_050000.pt: 21%|██ | 487MB / 2.35GB  ...nts/optim_050000_rank0.pt: 69%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_050000_rank2.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_050000_rank3.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_050000_rank4.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_050000_rank5.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_050000_rank6.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_050000_rank7.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_050000_rank1.pt: 31%|███ | 765MB / 2.49GB  Processing Files (0 / 9) : 60%|█████▉ | 13.3GB / 22.2GB, 998MB/s New Data Upload : 68%|██████▊ | 3.23GB / 4.74GB, 212MB/s  ...eckpoints/model_050000.pt: 21%|██▏ | 499MB / 2.35GB  ...nts/optim_050000_rank0.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_050000_rank2.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_050000_rank3.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_050000_rank4.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_050000_rank5.pt: 69%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_050000_rank6.pt: 71%|███████ | 1.75GB / 2.49GB  ...nts/optim_050000_rank7.pt: 69%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_050000_rank1.pt: 40%|███▉ | 984MB / 2.49GB  Processing Files (0 / 9) : 61%|██████ | 13.6GB / 22.2GB, 1.01GB/s New Data Upload : 69%|██████▉ | 3.27GB / 4.74GB, 212MB/s  ...eckpoints/model_050000.pt: 22%|██▏ | 509MB / 2.35GB  ...nts/optim_050000_rank0.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_050000_rank2.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_050000_rank3.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_050000_rank4.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_050000_rank5.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_050000_rank6.pt: 71%|███████▏ | 1.77GB / 2.49GB  ...nts/optim_050000_rank7.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_050000_rank1.pt: 44%|████▎ | 1.09GB / 2.49GB  Processing Files (0 / 9) : 62%|██████▏ | 13.9GB / 22.2GB, 1.02GB/s New Data Upload : 70%|███████ | 3.32GB / 4.74GB, 213MB/s  ...eckpoints/model_050000.pt: 22%|██▏ | 525MB / 2.35GB  ...nts/optim_050000_rank0.pt: 71%|███████▏ | 1.77GB / 2.49GB  ...nts/optim_050000_rank2.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_050000_rank3.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_050000_rank4.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_050000_rank5.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_050000_rank6.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_050000_rank7.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_050000_rank1.pt: 46%|████▌ | 1.14GB / 2.49GB  Processing Files (0 / 9) : 63%|██████▎ | 14.0GB / 22.2GB, 1.02GB/s New Data Upload : 71%|███████ | 3.37GB / 4.74GB, 215MB/s  ...eckpoints/model_050000.pt: 23%|██▎ | 540MB / 2.35GB  ...nts/optim_050000_rank0.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_050000_rank2.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_050000_rank3.pt: 73%|███████▎ | 1.80GB / 2.49GB  ...nts/optim_050000_rank4.pt: 71%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_050000_rank5.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_050000_rank6.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_050000_rank7.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_050000_rank1.pt: 52%|█████▏ | 1.28GB / 2.49GB  Processing Files (0 / 9) : 64%|██████▍ | 14.3GB / 22.2GB, 1.02GB/s New Data Upload : 72%|███████▏ | 3.42GB / 4.74GB, 214MB/s  ...eckpoints/model_050000.pt: 23%|██▎ | 546MB / 2.35GB  ...nts/optim_050000_rank0.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_050000_rank2.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_050000_rank3.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_050000_rank4.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_050000_rank5.pt: 73%|███████▎ | 1.80GB / 2.49GB  ...nts/optim_050000_rank6.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_050000_rank7.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_050000_rank1.pt: 57%|█████▋ | 1.41GB / 2.49GB  Processing Files (0 / 9) : 66%|██████▌ | 14.6GB / 22.2GB, 1.03GB/s New Data Upload : 73%|███████▎ | 3.46GB / 4.74GB, 214MB/s  ...eckpoints/model_050000.pt: 24%|██▍ | 558MB / 2.35GB  ...nts/optim_050000_rank0.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_050000_rank2.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_050000_rank3.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_050000_rank4.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_050000_rank5.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_050000_rank6.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_050000_rank7.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_050000_rank1.pt: 59%|█████▉ | 1.46GB / 2.49GB  Processing Files (0 / 9) : 66%|██████▌ | 14.7GB / 22.2GB, 1.02GB/s New Data Upload : 73%|███████▎ | 3.50GB / 4.80GB, 214MB/s  ...eckpoints/model_050000.pt: 24%|██▍ | 572MB / 2.35GB  ...nts/optim_050000_rank0.pt: 74%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_050000_rank2.pt: 74%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_050000_rank3.pt: 75%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_050000_rank4.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_050000_rank5.pt: 74%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_050000_rank6.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_050000_rank7.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_050000_rank1.pt: 60%|█████▉ | 1.48GB / 2.49GB  Processing Files (0 / 9) : 67%|██████▋ | 14.9GB / 22.2GB, 1.01GB/s New Data Upload : 74%|███████▎ | 3.54GB / 4.80GB, 214MB/s  ...eckpoints/model_050000.pt: 25%|██▍ | 580MB / 2.35GB  ...nts/optim_050000_rank0.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_050000_rank2.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_050000_rank3.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_050000_rank4.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_050000_rank5.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_050000_rank6.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_050000_rank7.pt: 73%|███████▎ | 1.80GB / 2.49GB  ...nts/optim_050000_rank1.pt: 65%|██████▍ | 1.61GB / 2.49GB  Processing Files (0 / 9) : 68%|██████▊ | 15.1GB / 22.2GB, 1.02GB/s New Data Upload : 74%|███████▎ | 3.59GB / 4.87GB, 214MB/s  ...eckpoints/model_050000.pt: 25%|██▌ | 595MB / 2.35GB  ...nts/optim_050000_rank0.pt: 75%|███████▌ | 1.86GB / 2.49GB  ...nts/optim_050000_rank2.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_050000_rank3.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_050000_rank4.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_050000_rank5.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_050000_rank6.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_050000_rank7.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_050000_rank1.pt: 66%|██████▌ | 1.63GB / 2.49GB  Processing Files (0 / 9) : 69%|██████▊ | 15.2GB / 22.2GB, 1.01GB/s New Data Upload : 74%|███████▎ | 3.63GB / 4.94GB, 214MB/s  ...eckpoints/model_050000.pt: 26%|██▌ | 608MB / 2.35GB  ...nts/optim_050000_rank0.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_050000_rank2.pt: 75%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_050000_rank3.pt: 77%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_050000_rank4.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_050000_rank5.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_050000_rank6.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_050000_rank7.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_050000_rank1.pt: 71%|███████ | 1.77GB / 2.49GB  Processing Files (0 / 9) : 70%|██████▉ | 15.5GB / 22.2GB, 1.02GB/s New Data Upload : 74%|███████▍ | 3.67GB / 4.94GB, 214MB/s  ...eckpoints/model_050000.pt: 26%|██▋ | 618MB / 2.35GB  ...nts/optim_050000_rank0.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_050000_rank2.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_050000_rank3.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_050000_rank4.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_050000_rank5.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_050000_rank6.pt: 77%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_050000_rank7.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_050000_rank1.pt: 73%|███████▎ | 1.82GB / 2.49GB  Processing Files (0 / 9) : 70%|███████ | 15.7GB / 22.2GB, 1.01GB/s New Data Upload : 74%|███████▍ | 3.70GB / 5.00GB, 213MB/s  ...eckpoints/model_050000.pt: 27%|██▋ | 631MB / 2.35GB  ...nts/optim_050000_rank0.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_050000_rank2.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_050000_rank3.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_050000_rank4.pt: 77%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_050000_rank5.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_050000_rank6.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_050000_rank7.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_050000_rank1.pt: 76%|███████▌ | 1.89GB / 2.49GB  Processing Files (0 / 9) : 71%|███████▏ | 15.9GB / 22.2GB, 1.00GB/s New Data Upload : 75%|███████▍ | 3.74GB / 5.00GB, 212MB/s  ...eckpoints/model_050000.pt: 27%|██▋ | 643MB / 2.35GB  ...nts/optim_050000_rank0.pt: 77%|███████▋ | 1.93GB / 2.49GB  ...nts/optim_050000_rank2.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_050000_rank3.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_050000_rank4.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_050000_rank5.pt: 77%|███████▋ | 1.93GB / 2.49GB  ...nts/optim_050000_rank6.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_050000_rank7.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_050000_rank1.pt: 77%|███████▋ | 1.91GB / 2.49GB  Processing Files (0 / 9) : 72%|███████▏ | 16.0GB / 22.2GB, 994MB/s New Data Upload : 75%|███████▍ | 3.78GB / 5.07GB, 212MB/s  ...eckpoints/model_050000.pt: 28%|██▊ | 661MB / 2.35GB  ...nts/optim_050000_rank0.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_050000_rank2.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_050000_rank3.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_050000_rank4.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_050000_rank5.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_050000_rank6.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_050000_rank7.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_050000_rank1.pt: 77%|███████▋ | 1.92GB / 2.49GB  Processing Files (0 / 9) : 72%|███████▏ | 16.1GB / 22.2GB, 985MB/s New Data Upload : 75%|███████▌ | 3.82GB / 5.07GB, 212MB/s  ...eckpoints/model_050000.pt: 29%|██▊ | 675MB / 2.35GB  ...nts/optim_050000_rank0.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_050000_rank2.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_050000_rank3.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_050000_rank4.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_050000_rank5.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_050000_rank6.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_050000_rank7.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_050000_rank1.pt: 78%|███████▊ | 1.94GB / 2.49GB  Processing Files (0 / 9) : 73%|███████▎ | 16.2GB / 22.2GB, 990MB/s New Data Upload : 75%|███████▌ | 3.86GB / 5.14GB, 214MB/s  ...eckpoints/model_050000.pt: 29%|██▉ | 689MB / 2.35GB  ...nts/optim_050000_rank0.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_050000_rank2.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_050000_rank3.pt: 79%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_050000_rank4.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_050000_rank5.pt: 79%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_050000_rank6.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_050000_rank7.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_050000_rank1.pt: 79%|███████▊ | 1.96GB / 2.49GB  Processing Files (0 / 9) : 74%|███████▎ | 16.4GB / 22.2GB, 984MB/s New Data Upload : 76%|███████▌ | 3.90GB / 5.14GB, 215MB/s  ...eckpoints/model_050000.pt: 30%|██▉ | 699MB / 2.35GB  ...nts/optim_050000_rank0.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_050000_rank2.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_050000_rank3.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_050000_rank4.pt: 79%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_050000_rank5.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_050000_rank6.pt: 80%|███████▉ | 1.99GB / 2.49GB  ...nts/optim_050000_rank7.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_050000_rank1.pt: 79%|███████▉ | 1.97GB / 2.49GB  Processing Files (0 / 9) : 74%|███████▍ | 16.5GB / 22.2GB, 979MB/s New Data Upload : 76%|███████▌ | 3.93GB / 5.21GB, 214MB/s  ...eckpoints/model_050000.pt: 31%|███ | 720MB / 2.35GB  ...nts/optim_050000_rank0.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_050000_rank2.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_050000_rank3.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_050000_rank4.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_050000_rank5.pt: 81%|████████ | 2.00GB / 2.49GB  ...nts/optim_050000_rank6.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_050000_rank7.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_050000_rank1.pt: 80%|███████▉ | 1.98GB / 2.49GB  Processing Files (0 / 9) : 75%|███████▍ | 16.6GB / 22.2GB, 975MB/s New Data Upload : 75%|███████▌ | 3.98GB / 5.27GB, 215MB/s  ...eckpoints/model_050000.pt: 31%|███▏ | 737MB / 2.35GB  ...nts/optim_050000_rank0.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_050000_rank2.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_050000_rank3.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_050000_rank4.pt: 81%|████████ | 2.00GB / 2.49GB  ...nts/optim_050000_rank5.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_050000_rank6.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_050000_rank7.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_050000_rank1.pt: 80%|████████ | 2.00GB / 2.49GB  Processing Files (0 / 9) : 75%|███████▌ | 16.7GB / 22.2GB, 967MB/s New Data Upload : 75%|███████▌ | 4.01GB / 5.34GB, 215MB/s  ...eckpoints/model_050000.pt: 32%|███▏ | 762MB / 2.35GB  ...nts/optim_050000_rank0.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_050000_rank2.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_050000_rank3.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_050000_rank4.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_050000_rank5.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_050000_rank6.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_050000_rank7.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_050000_rank1.pt: 81%|████████ | 2.01GB / 2.49GB  Processing Files (0 / 9) : 76%|███████▌ | 16.8GB / 22.2GB, 961MB/s New Data Upload : 75%|███████▍ | 4.05GB / 5.41GB, 215MB/s  ...eckpoints/model_050000.pt: 33%|███▎ | 780MB / 2.35GB  ...nts/optim_050000_rank0.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_050000_rank2.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_050000_rank3.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_050000_rank4.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_050000_rank5.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_050000_rank6.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_050000_rank7.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_050000_rank1.pt: 81%|████████ | 2.02GB / 2.49GB  Processing Files (0 / 9) : 76%|███████▌ | 16.9GB / 22.2GB, 952MB/s New Data Upload : 75%|███████▍ | 4.08GB / 5.47GB, 215MB/s  ...eckpoints/model_050000.pt: 34%|███▍ | 803MB / 2.35GB  ...nts/optim_050000_rank0.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_050000_rank2.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_050000_rank3.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_050000_rank4.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_050000_rank5.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_050000_rank6.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_050000_rank7.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_050000_rank1.pt: 82%|████████▏ | 2.03GB / 2.49GB  Processing Files (0 / 9) : 77%|███████▋ | 17.0GB / 22.2GB, 946MB/s New Data Upload : 74%|███████▍ | 4.12GB / 5.54GB, 215MB/s  ...eckpoints/model_050000.pt: 35%|███▌ | 823MB / 2.35GB  ...nts/optim_050000_rank0.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_050000_rank2.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_050000_rank3.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_050000_rank4.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_050000_rank5.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_050000_rank6.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_050000_rank7.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_050000_rank1.pt: 82%|████████▏ | 2.04GB / 2.49GB  Processing Files (0 / 9) : 77%|███████▋ | 17.2GB / 22.2GB, 943MB/s New Data Upload : 75%|███████▌ | 4.16GB / 5.54GB, 215MB/s  ...eckpoints/model_050000.pt: 36%|███▌ | 850MB / 2.35GB  ...nts/optim_050000_rank0.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_050000_rank2.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_050000_rank3.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_050000_rank4.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_050000_rank5.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_050000_rank6.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_050000_rank7.pt: 80%|███████▉ | 1.99GB / 2.49GB  ...nts/optim_050000_rank1.pt: 83%|████████▎ | 2.05GB / 2.49GB  Processing Files (0 / 9) : 78%|███████▊ | 17.3GB / 22.2GB, 933MB/s New Data Upload : 75%|███████▍ | 4.20GB / 5.61GB, 214MB/s  ...eckpoints/model_050000.pt: 37%|███▋ | 877MB / 2.35GB  ...nts/optim_050000_rank0.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_050000_rank2.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_050000_rank3.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_050000_rank4.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_050000_rank5.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_050000_rank6.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_050000_rank7.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_050000_rank1.pt: 83%|████████▎ | 2.06GB / 2.49GB  Processing Files (0 / 9) : 78%|███████▊ | 17.4GB / 22.2GB, 925MB/s New Data Upload : 76%|███████▌ | 4.24GB / 5.61GB, 214MB/s  ...eckpoints/model_050000.pt: 39%|███▊ | 905MB / 2.35GB  ...nts/optim_050000_rank0.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_050000_rank2.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_050000_rank3.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_050000_rank4.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_050000_rank5.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_050000_rank6.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_050000_rank7.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_050000_rank1.pt: 84%|████████▎ | 2.08GB / 2.49GB  Processing Files (0 / 9) : 79%|███████▉ | 17.5GB / 22.2GB, 915MB/s New Data Upload : 76%|███████▌ | 4.29GB / 5.68GB, 213MB/s  ...eckpoints/model_050000.pt: 40%|████ | 945MB / 2.35GB  ...nts/optim_050000_rank0.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_050000_rank2.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_050000_rank3.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_050000_rank4.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_050000_rank5.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_050000_rank6.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_050000_rank7.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_050000_rank1.pt: 84%|████████▍ | 2.09GB / 2.49GB  Processing Files (0 / 9) : 79%|███████▉ | 17.7GB / 22.2GB, 904MB/s New Data Upload : 76%|███████▌ | 4.34GB / 5.74GB, 213MB/s  ...eckpoints/model_050000.pt: 42%|████▏ | 982MB / 2.35GB  ...nts/optim_050000_rank0.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_050000_rank2.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_050000_rank3.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_050000_rank4.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_050000_rank5.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_050000_rank6.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_050000_rank7.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_050000_rank1.pt: 84%|████████▍ | 2.10GB / 2.49GB  Processing Files (0 / 9) : 80%|███████▉ | 17.8GB / 22.2GB, 895MB/s New Data Upload : 76%|███████▋ | 4.39GB / 5.74GB, 213MB/s  ...eckpoints/model_050000.pt: 43%|████▎ | 1.02GB / 2.35GB  ...nts/optim_050000_rank0.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_050000_rank2.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_050000_rank3.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_050000_rank4.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_050000_rank5.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_050000_rank6.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_050000_rank7.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_050000_rank1.pt: 85%|████████▍ | 2.11GB / 2.49GB  Processing Files (0 / 9) : 81%|████████ | 17.9GB / 22.2GB, 887MB/s New Data Upload : 76%|███████▋ | 4.44GB / 5.81GB, 214MB/s  ...eckpoints/model_050000.pt: 45%|████▍ | 1.05GB / 2.35GB  ...nts/optim_050000_rank0.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_050000_rank2.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_050000_rank3.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_050000_rank4.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_050000_rank5.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_050000_rank6.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_050000_rank7.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_050000_rank1.pt: 85%|████████▌ | 2.12GB / 2.49GB  Processing Files (0 / 9) : 81%|████████ | 18.0GB / 22.2GB, 880MB/s New Data Upload : 77%|███████▋ | 4.49GB / 5.81GB, 213MB/s  ...eckpoints/model_050000.pt: 46%|████▋ | 1.09GB / 2.35GB  ...nts/optim_050000_rank0.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_050000_rank2.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_050000_rank3.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_050000_rank4.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_050000_rank5.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_050000_rank6.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_050000_rank7.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_050000_rank1.pt: 86%|████████▌ | 2.13GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.1GB / 22.2GB, 877MB/s New Data Upload : 77%|███████▋ | 4.54GB / 5.88GB, 215MB/s  ...eckpoints/model_050000.pt: 48%|████▊ | 1.12GB / 2.35GB  ...nts/optim_050000_rank0.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_050000_rank2.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_050000_rank3.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_050000_rank4.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_050000_rank5.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_050000_rank6.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_050000_rank7.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_050000_rank1.pt: 86%|████████▌ | 2.14GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.3GB / 22.2GB, 868MB/s New Data Upload : 77%|███████▋ | 4.58GB / 5.94GB, 214MB/s  ...eckpoints/model_050000.pt: 49%|████▉ | 1.16GB / 2.35GB  ...nts/optim_050000_rank0.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_050000_rank2.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_050000_rank3.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_050000_rank4.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_050000_rank5.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_050000_rank6.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_050000_rank7.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_050000_rank1.pt: 86%|████████▋ | 2.15GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.4GB / 22.2GB, 858MB/s New Data Upload : 77%|███████▋ | 4.63GB / 6.01GB, 214MB/s  ...eckpoints/model_050000.pt: 51%|█████ | 1.19GB / 2.35GB  ...nts/optim_050000_rank0.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_050000_rank2.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_050000_rank3.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_050000_rank4.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_050000_rank5.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_050000_rank6.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_050000_rank7.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_050000_rank1.pt: 87%|████████▋ | 2.16GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.5GB / 22.2GB, 851MB/s New Data Upload : 78%|███████▊ | 4.68GB / 6.01GB, 214MB/s  ...eckpoints/model_050000.pt: 52%|█████▏ | 1.23GB / 2.35GB  ...nts/optim_050000_rank0.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_050000_rank2.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_050000_rank3.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_050000_rank4.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_050000_rank5.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_050000_rank6.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_050000_rank7.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_050000_rank1.pt: 87%|████████▋ | 2.16GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▎ | 18.6GB / 22.2GB, 842MB/s New Data Upload : 78%|███████▊ | 4.72GB / 6.08GB, 214MB/s  ...eckpoints/model_050000.pt: 54%|█████▍ | 1.27GB / 2.35GB  ...nts/optim_050000_rank0.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_050000_rank2.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_050000_rank3.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_050000_rank4.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_050000_rank5.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_050000_rank6.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_050000_rank7.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_050000_rank1.pt: 87%|████████▋ | 2.17GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▍ | 18.7GB / 22.2GB, 835MB/s New Data Upload : 78%|███████▊ | 4.77GB / 6.14GB, 214MB/s  ...eckpoints/model_050000.pt: 56%|█████▌ | 1.31GB / 2.35GB  ...nts/optim_050000_rank0.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_050000_rank2.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_050000_rank3.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_050000_rank4.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_050000_rank5.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_050000_rank6.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_050000_rank7.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_050000_rank1.pt: 88%|████████▊ | 2.18GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▍ | 18.8GB / 22.2GB, 829MB/s New Data Upload : 79%|███████▊ | 4.83GB / 6.14GB, 215MB/s  ...eckpoints/model_050000.pt: 57%|█████▋ | 1.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_050000_rank2.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_050000_rank3.pt: 89%|████████▊ | 2.21GB / 2.49GB  ...nts/optim_050000_rank4.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_050000_rank5.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_050000_rank6.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_050000_rank7.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_050000_rank1.pt: 88%|████████▊ | 2.19GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▌ | 18.9GB / 22.2GB, 816MB/s New Data Upload : 78%|███████▊ | 4.87GB / 6.21GB, 214MB/s  ...eckpoints/model_050000.pt: 59%|█████▉ | 1.38GB / 2.35GB  ...nts/optim_050000_rank0.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_050000_rank2.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_050000_rank3.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_050000_rank4.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_050000_rank5.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_050000_rank6.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_050000_rank7.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_050000_rank1.pt: 88%|████████▊ | 2.20GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▌ | 19.0GB / 22.2GB, 810MB/s New Data Upload : 79%|███████▉ | 4.92GB / 6.21GB, 215MB/s  ...eckpoints/model_050000.pt: 60%|██████ | 1.42GB / 2.35GB  ...nts/optim_050000_rank0.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_050000_rank2.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_050000_rank3.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_050000_rank4.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_050000_rank5.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_050000_rank6.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_050000_rank7.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_050000_rank1.pt: 89%|████████▉ | 2.21GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▌ | 19.1GB / 22.2GB, 813MB/s New Data Upload : 79%|███████▉ | 4.97GB / 6.28GB, 216MB/s  ...eckpoints/model_050000.pt: 62%|██████▏ | 1.45GB / 2.35GB  ...nts/optim_050000_rank0.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_050000_rank2.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_050000_rank3.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_050000_rank4.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_050000_rank5.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_050000_rank6.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_050000_rank7.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_050000_rank1.pt: 89%|████████▉ | 2.21GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▋ | 19.2GB / 22.2GB, 807MB/s New Data Upload : 80%|███████▉ | 5.01GB / 6.29GB, 217MB/s  ...eckpoints/model_050000.pt: 64%|██████▎ | 1.49GB / 2.35GB  ...nts/optim_050000_rank0.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_050000_rank2.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_050000_rank3.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_050000_rank4.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_050000_rank5.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_050000_rank6.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_050000_rank7.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_050000_rank1.pt: 89%|████████▉ | 2.22GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.3GB / 22.2GB, 797MB/s New Data Upload : 81%|████████ | 5.06GB / 6.29GB, 217MB/s  ...eckpoints/model_050000.pt: 65%|██████▌ | 1.53GB / 2.35GB  ...nts/optim_050000_rank0.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_050000_rank2.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_050000_rank3.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_050000_rank4.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_050000_rank5.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_050000_rank6.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_050000_rank7.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_050000_rank1.pt: 90%|████████▉ | 2.23GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.4GB / 22.2GB, 793MB/s New Data Upload : 81%|████████ | 5.11GB / 6.29GB, 218MB/s  ...eckpoints/model_050000.pt: 67%|██████▋ | 1.57GB / 2.35GB  ...nts/optim_050000_rank0.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_050000_rank2.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_050000_rank3.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_050000_rank4.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_050000_rank5.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_050000_rank6.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_050000_rank7.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_050000_rank1.pt: 90%|████████▉ | 2.24GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.5GB / 22.2GB, 787MB/s New Data Upload : 82%|████████▏ | 5.15GB / 6.29GB, 218MB/s  ...eckpoints/model_050000.pt: 68%|██████▊ | 1.61GB / 2.35GB  ...nts/optim_050000_rank0.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_050000_rank2.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_050000_rank3.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_050000_rank4.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_050000_rank5.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_050000_rank6.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_050000_rank7.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_050000_rank1.pt: 90%|█████████ | 2.24GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.6GB / 22.2GB, 768MB/s New Data Upload : 83%|████████▎ | 5.20GB / 6.29GB, 219MB/s  ...eckpoints/model_050000.pt: 69%|██████▉ | 1.63GB / 2.35GB  ...nts/optim_050000_rank0.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_050000_rank2.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_050000_rank3.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_050000_rank4.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_050000_rank5.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_050000_rank6.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_050000_rank7.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_050000_rank1.pt: 91%|█████████ | 2.25GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.7GB / 22.2GB, 750MB/s New Data Upload : 83%|████████▎ | 5.24GB / 6.29GB, 218MB/s  ...eckpoints/model_050000.pt: 71%|███████ | 1.67GB / 2.35GB  ...nts/optim_050000_rank0.pt: 92%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_050000_rank2.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_050000_rank3.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_050000_rank4.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_050000_rank5.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_050000_rank6.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_050000_rank7.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_050000_rank1.pt: 91%|█████████ | 2.26GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▉ | 19.8GB / 22.2GB, 732MB/s New Data Upload : 84%|████████▍ | 5.29GB / 6.29GB, 219MB/s  ...eckpoints/model_050000.pt: 73%|███████▎ | 1.70GB / 2.35GB  ...nts/optim_050000_rank0.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_050000_rank2.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_050000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_050000_rank4.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_050000_rank5.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_050000_rank6.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_050000_rank7.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_050000_rank1.pt: 91%|█████████ | 2.26GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▉ | 19.8GB / 22.2GB, 720MB/s New Data Upload : 85%|████████▍ | 5.33GB / 6.29GB, 219MB/s  ...eckpoints/model_050000.pt: 74%|███████▍ | 1.74GB / 2.35GB  ...nts/optim_050000_rank0.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_050000_rank2.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_050000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_050000_rank4.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_050000_rank5.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_050000_rank6.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_050000_rank7.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_050000_rank1.pt: 91%|█████████▏| 2.27GB / 2.49GB  Processing Files (0 / 9) : 90%|████████▉ | 20.0GB / 22.2GB, 699MB/s New Data Upload : 86%|████████▌ | 5.38GB / 6.29GB, 219MB/s  ...eckpoints/model_050000.pt: 75%|███████▌ | 1.77GB / 2.35GB  ...nts/optim_050000_rank0.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_050000_rank2.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_050000_rank3.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_050000_rank4.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_050000_rank5.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_050000_rank6.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_050000_rank7.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_050000_rank1.pt: 92%|█████████▏| 2.28GB / 2.49GB  Processing Files (0 / 9) : 90%|█████████ | 20.0GB / 22.2GB, 674MB/s New Data Upload : 86%|████████▌ | 5.42GB / 6.29GB, 219MB/s  ...eckpoints/model_050000.pt: 77%|███████▋ | 1.80GB / 2.35GB  ...nts/optim_050000_rank0.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_050000_rank2.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_050000_rank3.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_050000_rank4.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_050000_rank5.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_050000_rank6.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_050000_rank7.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_050000_rank1.pt: 92%|█████████▏| 2.29GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.1GB / 22.2GB, 674MB/s New Data Upload : 87%|████████▋ | 5.46GB / 6.29GB, 219MB/s  ...eckpoints/model_050000.pt: 78%|███████▊ | 1.84GB / 2.35GB  ...nts/optim_050000_rank0.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_050000_rank2.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_050000_rank3.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_050000_rank4.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_050000_rank5.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_050000_rank6.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_050000_rank7.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_050000_rank1.pt: 92%|█████████▏| 2.30GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.2GB / 22.2GB, 649MB/s New Data Upload : 88%|████████▊ | 5.50GB / 6.29GB, 218MB/s  ...eckpoints/model_050000.pt: 79%|███████▉ | 1.86GB / 2.35GB  ...nts/optim_050000_rank0.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_050000_rank2.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_050000_rank3.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_050000_rank4.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_050000_rank5.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_050000_rank6.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_050000_rank7.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_050000_rank1.pt: 93%|█████████▎| 2.30GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████▏| 20.3GB / 22.2GB, 633MB/s New Data Upload : 88%|████████▊ | 5.54GB / 6.29GB, 218MB/s  ...eckpoints/model_050000.pt: 80%|████████ | 1.88GB / 2.35GB  ...nts/optim_050000_rank0.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_050000_rank2.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_050000_rank3.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_050000_rank4.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_050000_rank5.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_050000_rank6.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_050000_rank7.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_050000_rank1.pt: 93%|█████████▎| 2.31GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.4GB / 22.2GB, 623MB/s New Data Upload : 89%|████████▊ | 5.57GB / 6.29GB, 216MB/s  ...eckpoints/model_050000.pt: 81%|████████ | 1.90GB / 2.35GB  ...nts/optim_050000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_050000_rank2.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_050000_rank3.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_050000_rank4.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_050000_rank5.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_050000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_050000_rank7.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_050000_rank1.pt: 93%|█████████▎| 2.32GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.5GB / 22.2GB, 606MB/s New Data Upload : 89%|████████▉ | 5.60GB / 6.29GB, 214MB/s  ...eckpoints/model_050000.pt: 82%|████████▏ | 1.93GB / 2.35GB  ...nts/optim_050000_rank0.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_050000_rank2.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_050000_rank3.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_050000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_050000_rank5.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_050000_rank6.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_050000_rank7.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_050000_rank1.pt: 94%|█████████▍| 2.33GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.6GB / 22.2GB, 592MB/s New Data Upload : 90%|████████▉ | 5.64GB / 6.29GB, 214MB/s  ...eckpoints/model_050000.pt: 83%|████████▎ | 1.94GB / 2.35GB  ...nts/optim_050000_rank0.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_050000_rank2.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_050000_rank3.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_050000_rank4.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_050000_rank5.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_050000_rank6.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_050000_rank7.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_050000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.7GB / 22.2GB, 585MB/s New Data Upload : 90%|█████████ | 5.67GB / 6.29GB, 212MB/s  ...eckpoints/model_050000.pt: 83%|████████▎ | 1.96GB / 2.35GB  ...nts/optim_050000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_050000_rank2.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_050000_rank3.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_050000_rank4.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_050000_rank5.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_050000_rank6.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_050000_rank7.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_050000_rank1.pt: 95%|█████████▍| 2.35GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▎| 20.8GB / 22.2GB, 582MB/s New Data Upload : 91%|█████████ | 5.70GB / 6.29GB, 211MB/s  ...eckpoints/model_050000.pt: 84%|████████▍ | 1.98GB / 2.35GB  ...nts/optim_050000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_050000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_050000_rank3.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_050000_rank4.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_050000_rank5.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_050000_rank6.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_050000_rank7.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_050000_rank1.pt: 95%|█████████▍| 2.36GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 20.9GB / 22.2GB, 563MB/s New Data Upload : 91%|█████████ | 5.72GB / 6.29GB, 210MB/s  ...eckpoints/model_050000.pt: 85%|████████▌ | 2.00GB / 2.35GB  ...nts/optim_050000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_050000_rank2.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_050000_rank3.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_050000_rank4.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_050000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_050000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_050000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_050000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 21.0GB / 22.2GB, 561MB/s New Data Upload : 92%|█████████▏| 5.76GB / 6.29GB, 209MB/s  ...eckpoints/model_050000.pt: 86%|████████▌ | 2.01GB / 2.35GB  ...nts/optim_050000_rank0.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_050000_rank2.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_050000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_050000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_050000_rank5.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_050000_rank6.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_050000_rank7.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_050000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▍| 21.0GB / 22.2GB, 544MB/s New Data Upload : 92%|█████████▏| 5.78GB / 6.29GB, 207MB/s  ...eckpoints/model_050000.pt: 86%|████████▌ | 2.02GB / 2.35GB  ...nts/optim_050000_rank0.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_050000_rank2.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_050000_rank3.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_050000_rank4.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_050000_rank5.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_050000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_050000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_050000_rank1.pt: 96%|█████████▌| 2.39GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.1GB / 22.2GB, 536MB/s New Data Upload : 92%|█████████▏| 5.80GB / 6.29GB, 206MB/s  ...eckpoints/model_050000.pt: 87%|████████▋ | 2.04GB / 2.35GB  ...nts/optim_050000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_050000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_050000_rank3.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_050000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_050000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_050000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_050000_rank7.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_050000_rank1.pt: 96%|█████████▋| 2.39GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.2GB / 22.2GB, 527MB/s New Data Upload : 93%|█████████▎| 5.83GB / 6.29GB, 204MB/s  ...eckpoints/model_050000.pt: 87%|████████▋ | 2.05GB / 2.35GB  ...nts/optim_050000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_050000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_050000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_050000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_050000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_050000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_050000_rank7.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_050000_rank1.pt: 97%|█████████▋| 2.40GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 522MB/s New Data Upload : 93%|█████████▎| 5.85GB / 6.29GB, 203MB/s  ...eckpoints/model_050000.pt: 88%|████████▊ | 2.06GB / 2.35GB  ...nts/optim_050000_rank0.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_050000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_050000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_050000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_050000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_050000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_050000_rank7.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_050000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.4GB / 22.2GB, 517MB/s New Data Upload : 93%|█████████▎| 5.87GB / 6.29GB, 201MB/s  ...eckpoints/model_050000.pt: 89%|████████▊ | 2.08GB / 2.35GB  ...nts/optim_050000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_050000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_050000_rank3.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_050000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_050000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_050000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_050000_rank7.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_050000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▋| 21.4GB / 22.2GB, 511MB/s New Data Upload : 94%|█████████▍| 5.90GB / 6.29GB, 200MB/s  ...eckpoints/model_050000.pt: 89%|████████▉ | 2.09GB / 2.35GB  ...nts/optim_050000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_050000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_050000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_050000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_050000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_050000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_050000_rank7.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_050000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 504MB/s New Data Upload : 94%|█████████▍| 5.92GB / 6.29GB, 198MB/s  ...eckpoints/model_050000.pt: 90%|████████▉ | 2.10GB / 2.35GB  ...nts/optim_050000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_050000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_050000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_050000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_050000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_050000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_050000_rank7.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_050000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 502MB/s New Data Upload : 95%|█████████▍| 5.95GB / 6.29GB, 198MB/s  ...eckpoints/model_050000.pt: 90%|█████████ | 2.12GB / 2.35GB  ...nts/optim_050000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_050000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_050000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_050000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_050000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_050000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_050000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 493MB/s New Data Upload : 95%|█████████▍| 5.97GB / 6.29GB, 195MB/s  ...eckpoints/model_050000.pt: 91%|█████████ | 2.13GB / 2.35GB  ...nts/optim_050000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_050000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_050000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_050000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_050000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_050000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 487MB/s New Data Upload : 95%|█████████▌| 5.99GB / 6.29GB, 194MB/s  ...eckpoints/model_050000.pt: 91%|█████████▏| 2.15GB / 2.35GB  ...nts/optim_050000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_050000_rank4.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_050000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_050000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_050000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 479MB/s New Data Upload : 96%|█████████▌| 6.01GB / 6.29GB, 192MB/s  ...eckpoints/model_050000.pt: 92%|█████████▏| 2.16GB / 2.35GB  ...nts/optim_050000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank3.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...nts/optim_050000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_050000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_050000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 475MB/s New Data Upload : 96%|█████████▌| 6.03GB / 6.29GB, 190MB/s  ...eckpoints/model_050000.pt: 92%|█████████▏| 2.17GB / 2.35GB  ...nts/optim_050000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_050000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 469MB/s New Data Upload : 96%|█████████▌| 6.04GB / 6.29GB, 188MB/s  ...eckpoints/model_050000.pt: 93%|█████████▎| 2.17GB / 2.35GB  ...nts/optim_050000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_050000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 459MB/s New Data Upload : 96%|█████████▋| 6.06GB / 6.29GB, 186MB/s  ...eckpoints/model_050000.pt: 93%|█████████▎| 2.19GB / 2.35GB  ...nts/optim_050000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_050000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 451MB/s New Data Upload : 97%|█████████▋| 6.07GB / 6.29GB, 184MB/s  ...eckpoints/model_050000.pt: 93%|█████████▎| 2.20GB / 2.35GB  ...nts/optim_050000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_050000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 442MB/s New Data Upload : 97%|█████████▋| 6.09GB / 6.29GB, 181MB/s  ...eckpoints/model_050000.pt: 94%|█████████▍| 2.20GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_050000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 22.0GB / 22.2GB, 432MB/s New Data Upload : 97%|█████████▋| 6.10GB / 6.29GB, 178MB/s  ...eckpoints/model_050000.pt: 94%|█████████▍| 2.21GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_050000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 423MB/s New Data Upload : 97%|█████████▋| 6.11GB / 6.29GB, 174MB/s  ...eckpoints/model_050000.pt: 95%|█████████▍| 2.22GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 416MB/s New Data Upload : 97%|█████████▋| 6.13GB / 6.29GB, 170MB/s  ...eckpoints/model_050000.pt: 95%|█████████▍| 2.23GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 403MB/s New Data Upload : 98%|█████████▊| 6.14GB / 6.29GB, 166MB/s  ...eckpoints/model_050000.pt: 95%|█████████▌| 2.24GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 396MB/s New Data Upload : 98%|█████████▊| 6.15GB / 6.29GB, 163MB/s  ...eckpoints/model_050000.pt: 95%|█████████▌| 2.24GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 386MB/s New Data Upload : 98%|█████████▊| 6.16GB / 6.29GB, 159MB/s  ...eckpoints/model_050000.pt: 96%|█████████▌| 2.25GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 377MB/s New Data Upload : 98%|█████████▊| 6.17GB / 6.29GB, 156MB/s  ...eckpoints/model_050000.pt: 96%|█████████▌| 2.26GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 369MB/s New Data Upload : 98%|█████████▊| 6.18GB / 6.29GB, 152MB/s  ...eckpoints/model_050000.pt: 96%|█████████▋| 2.26GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 358MB/s New Data Upload : 98%|█████████▊| 6.19GB / 6.29GB, 148MB/s  ...eckpoints/model_050000.pt: 96%|█████████▋| 2.26GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 350MB/s New Data Upload : 98%|█████████▊| 6.19GB / 6.29GB, 144MB/s  ...eckpoints/model_050000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (1 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 339MB/s New Data Upload : 99%|█████████▊| 6.20GB / 6.29GB, 140MB/s  ...eckpoints/model_050000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (1 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 327MB/s New Data Upload : 99%|█████████▊| 6.20GB / 6.29GB, 135MB/s  ...eckpoints/model_050000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (1 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 319MB/s New Data Upload : 99%|█████████▊| 6.21GB / 6.29GB, 131MB/s  ...eckpoints/model_050000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (5 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 310MB/s New Data Upload : 99%|█████████▉| 6.22GB / 6.29GB, 127MB/s  ...eckpoints/model_050000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (5 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 297MB/s New Data Upload : 99%|█████████▉| 6.22GB / 6.29GB, 123MB/s  ...eckpoints/model_050000.pt: 98%|█████████▊| 2.29GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (5 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 292MB/s New Data Upload : 99%|█████████▉| 6.23GB / 6.29GB, 119MB/s  ...eckpoints/model_050000.pt: 98%|█████████▊| 2.29GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (5 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 282MB/s New Data Upload : 99%|█████████▉| 6.23GB / 6.29GB, 115MB/s  ...eckpoints/model_050000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (5 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 275MB/s New Data Upload : 99%|█████████▉| 6.24GB / 6.29GB, 111MB/s  ...eckpoints/model_050000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (5 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 265MB/s New Data Upload : 99%|█████████▉| 6.24GB / 6.29GB, 107MB/s  ...eckpoints/model_050000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (5 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 257MB/s New Data Upload : 99%|█████████▉| 6.24GB / 6.29GB, 102MB/s  ...eckpoints/model_050000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 248MB/s New Data Upload : 99%|█████████▉| 6.25GB / 6.29GB, 98.8MB/s  ...eckpoints/model_050000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 238MB/s New Data Upload : 99%|█████████▉| 6.25GB / 6.29GB, 94.0MB/s  ...eckpoints/model_050000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 231MB/s New Data Upload : 99%|█████████▉| 6.25GB / 6.29GB, 90.4MB/s  ...eckpoints/model_050000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 219MB/s New Data Upload : 99%|█████████▉| 6.25GB / 6.29GB, 85.4MB/s  ...eckpoints/model_050000.pt: 99%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 211MB/s New Data Upload : 99%|█████████▉| 6.25GB / 6.29GB, 81.8MB/s  ...eckpoints/model_050000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 202MB/s New Data Upload : 99%|█████████▉| 6.25GB / 6.29GB, 78.1MB/s  ...eckpoints/model_050000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 193MB/s New Data Upload : 100%|█████████▉| 6.26GB / 6.29GB, 74.1MB/s  ...eckpoints/model_050000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 186MB/s New Data Upload : 100%|█████████▉| 6.26GB / 6.29GB, 70.9MB/s  ...eckpoints/model_050000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 177MB/s New Data Upload : 100%|█████████▉| 6.26GB / 6.29GB, 67.6MB/s  ...eckpoints/model_050000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 168MB/s New Data Upload : 100%|█████████▉| 6.26GB / 6.29GB, 64.9MB/s  ...eckpoints/model_050000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 157MB/s New Data Upload : 100%|█████████▉| 6.27GB / 6.29GB, 61.5MB/s  ...eckpoints/model_050000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 149MB/s New Data Upload : 100%|█████████▉| 6.27GB / 6.29GB, 58.9MB/s  ...eckpoints/model_050000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 140MB/s New Data Upload : 100%|█████████▉| 6.27GB / 6.29GB, 56.2MB/s  ...eckpoints/model_050000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 123MB/s New Data Upload : 100%|█████████▉| 6.27GB / 6.29GB, 50.3MB/s  ...eckpoints/model_050000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 115MB/s New Data Upload : 100%|█████████▉| 6.27GB / 6.29GB, 48.3MB/s  ...eckpoints/model_050000.pt: 99%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 106MB/s New Data Upload : 100%|█████████▉| 6.27GB / 6.29GB, 46.2MB/s  ...eckpoints/model_050000.pt: 99%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 98.0MB/s New Data Upload : 100%|█████████▉| 6.28GB / 6.29GB, 43.7MB/s  ...eckpoints/model_050000.pt: 99%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 91.3MB/s New Data Upload : 100%|█████████▉| 6.28GB / 6.29GB, 41.8MB/s  ...eckpoints/model_050000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 83.8MB/s New Data Upload : 100%|█████████▉| 6.28GB / 6.29GB, 39.5MB/s  ...eckpoints/model_050000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 76.3MB/s New Data Upload : 100%|█████████▉| 6.28GB / 6.29GB, 37.0MB/s  ...eckpoints/model_050000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 69.1MB/s New Data Upload : 100%|█████████▉| 6.28GB / 6.29GB, 35.1MB/s  ...eckpoints/model_050000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 61.8MB/s New Data Upload : 100%|█████████▉| 6.28GB / 6.29GB, 32.9MB/s  ...eckpoints/model_050000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 57.8MB/s New Data Upload : 100%|█████████▉| 6.28GB / 6.29GB, 30.8MB/s  ...eckpoints/model_050000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 49.6MB/s New Data Upload : 100%|█████████▉| 6.28GB / 6.29GB, 27.2MB/s  ...eckpoints/model_050000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 44.2MB/s New Data Upload : 100%|█████████▉| 6.29GB / 6.29GB, 25.4MB/s  ...eckpoints/model_050000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 39.8MB/s New Data Upload : 100%|█████████▉| 6.29GB / 6.29GB, 24.0MB/s  ...eckpoints/model_050000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (9 / 9) : 100%|██████████| 22.2GB / 22.2GB, 21.1MB/s New Data Upload : 100%|██████████| 6.29GB / 6.29GB, 15.8MB/s  ...eckpoints/model_050000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_050000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (9 / 9) : 100%|██████████| 22.2GB / 22.2GB, 6.49MB/s New Data Upload : 100%|██████████| 6.29GB / 6.29GB, 6.49MB/s ...eckpoints/model_050000.pt: 100%|██████████| 2.35GB / 2.35GB ...nts/optim_050000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_050000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_050000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_050000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_050000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_050000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_050000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_050000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB [15:59:00] Step 50000 uploaded [16:59:00] Uploading step 55000... Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...eckpoints/model_055000.pt: 0%| | 3.67MB / 2.35GB  ...eckpoints/model_055000.pt: 0%| | 3.67MB / 2.35GB  Processing Files (0 / 1) : 0%| | 3.67MB / 22.2GB, 4.59MB/s New Data Upload : 0%| | 587kB / 604MB, 734kB/s  ...eckpoints/model_055000.pt: 0%| | 3.67MB / 2.35GB  ...eckpoints/model_055000.pt: 0%| | 4.25MB / 2.35GB  Processing Files (0 / 1) : 0%| | 4.25MB / 22.2GB, 3.55MB/s New Data Upload : 0%| | 1.17MB / 604MB, 979kB/s  ...eckpoints/model_055000.pt: 0%| | 7.78MB / 2.35GB  Processing Files (0 / 1) : 0%| | 7.78MB / 22.2GB, 5.56MB/s New Data Upload : 1%| | 4.70MB / 604MB, 3.36MB/s  ...eckpoints/model_055000.pt: 0%| | 11.3MB / 2.35GB  Processing Files (0 / 1) : 0%| | 11.3MB / 22.2GB, 7.06MB/s New Data Upload : 1%|▏ | 8.22MB / 604MB, 5.14MB/s  ...eckpoints/model_055000.pt: 1%| | 15.4MB / 2.35GB  Processing Files (0 / 1) : 0%| | 15.4MB / 22.2GB, 8.56MB/s New Data Upload : 2%|▏ | 12.3MB / 604MB, 6.85MB/s  ...eckpoints/model_055000.pt: 1%| | 23.6MB / 2.35GB  Processing Files (0 / 1) : 0%| | 23.6MB / 22.2GB, 11.8MB/s New Data Upload : 3%|▎ | 20.5MB / 604MB, 10.3MB/s  ...eckpoints/model_055000.pt: 1%|▏ | 33.0MB / 2.35GB  Processing Files (0 / 1) : 0%| | 33.0MB / 22.2GB, 15.0MB/s New Data Upload : 5%|▍ | 29.9MB / 604MB, 13.6MB/s  ...eckpoints/model_055000.pt: 2%|▏ | 42.4MB / 2.35GB  Processing Files (0 / 1) : 0%| | 42.4MB / 22.2GB, 17.7MB/s New Data Upload : 7%|▋ | 39.3MB / 604MB, 16.4MB/s  ...eckpoints/model_055000.pt: 2%|▏ | 51.2MB / 2.35GB  Processing Files (0 / 1) : 0%| | 51.2MB / 22.2GB, 19.7MB/s New Data Upload : 8%|▊ | 48.1MB / 604MB, 18.5MB/s  ...eckpoints/model_055000.pt: 2%|▏ | 55.3MB / 2.35GB  Processing Files (0 / 1) : 0%| | 55.3MB / 22.2GB, 19.8MB/s New Data Upload : 9%|▊ | 52.3MB / 604MB, 18.7MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 64.7MB / 2.35GB  Processing Files (0 / 1) : 0%| | 64.7MB / 22.2GB, 21.6MB/s New Data Upload : 10%|█ | 61.6MB / 604MB, 20.5MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 70.0MB / 2.35GB  Processing Files (0 / 1) : 0%| | 70.0MB / 22.2GB, 21.9MB/s New Data Upload : 11%|█ | 66.9MB / 604MB, 20.9MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 70.0MB / 2.35GB  ...eckpoints/model_055000.pt: 3%|▎ | 70.0MB / 2.35GB  ...eckpoints/model_055000.pt: 3%|▎ | 70.0MB / 2.35GB  ...eckpoints/model_055000.pt: 3%|▎ | 70.2MB / 2.35GB  Processing Files (0 / 1) : 0%| | 70.2MB / 22.2GB, 17.5MB/s New Data Upload : 10%|▉ | 67.1MB / 671MB, 16.8MB/s  ...nts/optim_055000_rank1.pt: 0%| | 305kB / 2.49GB  ...nts/optim_055000_rank2.pt: 0%| | 305kB / 2.49GB  ...nts/optim_055000_rank3.pt: 0%| | 6.44MB / 2.49GB  ...nts/optim_055000_rank4.pt: 0%| | 305kB / 2.49GB  ...nts/optim_055000_rank5.pt: 0%| | 305kB / 2.49GB  ...nts/optim_055000_rank7.pt: 0%| | 890kB / 2.49GB  ...eckpoints/model_055000.pt: 3%|▎ | 70.2MB / 2.35GB  ...nts/optim_055000_rank1.pt: 0%| | 305kB / 2.49GB  ...nts/optim_055000_rank2.pt: 0%| | 305kB / 2.49GB  ...nts/optim_055000_rank3.pt: 0%| | 6.44MB / 2.49GB  ...nts/optim_055000_rank4.pt: 0%| | 305kB / 2.49GB  ...nts/optim_055000_rank5.pt: 0%| | 305kB / 2.49GB  ...nts/optim_055000_rank7.pt: 0%| | 890kB / 2.49GB  Processing Files (0 / 7) : 0%| | 78.7MB / 22.2GB, 18.7MB/s New Data Upload : 10%|█ | 74.1MB / 738MB, 17.6MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 70.2MB / 2.35GB  ...nts/optim_055000_rank1.pt: 0%| | 693kB / 2.49GB  ...nts/optim_055000_rank2.pt: 0%| | 693kB / 2.49GB  ...nts/optim_055000_rank3.pt: 1%| | 14.6MB / 2.49GB  ...nts/optim_055000_rank4.pt: 0%| | 693kB / 2.49GB  ...nts/optim_055000_rank5.pt: 0%| | 693kB / 2.49GB  ...nts/optim_055000_rank7.pt: 0%| | 1.28MB / 2.49GB  Processing Files (0 / 7) : 0%| | 88.8MB / 22.2GB, 20.2MB/s New Data Upload : 11%|█ | 82.3MB / 738MB, 18.7MB/s  ...nts/optim_055000_rank0.pt: 0%| | 2.31MB / 2.49GB  ...eckpoints/model_055000.pt: 3%|▎ | 70.2MB / 2.35GB  ...nts/optim_055000_rank1.pt: 0%| | 1.19MB / 2.49GB  ...nts/optim_055000_rank2.pt: 0%| | 1.19MB / 2.49GB  ...nts/optim_055000_rank3.pt: 1%| | 24.0MB / 2.49GB  ...nts/optim_055000_rank4.pt: 0%| | 1.19MB / 2.49GB  ...nts/optim_055000_rank5.pt: 0%| | 1.19MB / 2.49GB  ...nts/optim_055000_rank7.pt: 0%| | 2.36MB / 2.49GB  ...nts/optim_055000_rank0.pt: 0%| | 2.31MB / 2.49GB  Processing Files (0 / 8) : 0%| | 104MB / 22.2GB, 22.5MB/s New Data Upload : 13%|█▎ | 93.4MB / 738MB, 20.3MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 70.2MB / 2.35GB  ...nts/optim_055000_rank1.pt: 0%| | 1.41MB / 2.49GB  ...nts/optim_055000_rank2.pt: 0%| | 1.41MB / 2.49GB  ...nts/optim_055000_rank3.pt: 1%| | 28.7MB / 2.49GB  ...nts/optim_055000_rank4.pt: 0%| | 1.41MB / 2.49GB  ...nts/optim_055000_rank5.pt: 0%| | 1.41MB / 2.49GB  ...nts/optim_055000_rank7.pt: 0%| | 4.92MB / 2.49GB  ...nts/optim_055000_rank0.pt: 0%| | 2.53MB / 2.49GB  Processing Files (0 / 8) : 1%| | 112MB / 22.2GB, 23.3MB/s New Data Upload : 12%|█▏ | 100MB / 805MB, 20.9MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 70.2MB / 2.35GB  ...nts/optim_055000_rank1.pt: 0%| | 1.85MB / 2.49GB  ...nts/optim_055000_rank2.pt: 0%| | 1.85MB / 2.49GB  ...nts/optim_055000_rank3.pt: 2%|▏ | 38.1MB / 2.49GB  ...nts/optim_055000_rank4.pt: 0%| | 1.85MB / 2.49GB  ...nts/optim_055000_rank5.pt: 0%| | 1.85MB / 2.49GB  ...nts/optim_055000_rank7.pt: 0%| | 10.1MB / 2.49GB  ...nts/optim_055000_rank0.pt: 0%| | 2.97MB / 2.49GB  Processing Files (0 / 8) : 1%| | 129MB / 22.2GB, 25.7MB/s New Data Upload : 14%|█▍ | 114MB / 805MB, 22.9MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 70.2MB / 2.35GB  ...nts/optim_055000_rank1.pt: 0%| | 2.38MB / 2.49GB  ...nts/optim_055000_rank2.pt: 0%| | 2.38MB / 2.49GB  ...nts/optim_055000_rank3.pt: 2%|▏ | 47.4MB / 2.49GB  ...nts/optim_055000_rank4.pt: 0%| | 2.38MB / 2.49GB  ...nts/optim_055000_rank5.pt: 0%| | 2.38MB / 2.49GB  ...nts/optim_055000_rank7.pt: 1%| | 19.4MB / 2.49GB  ...nts/optim_055000_rank0.pt: 0%| | 5.17MB / 2.49GB  Processing Files (0 / 8) : 1%| | 152MB / 22.2GB, 29.2MB/s New Data Upload : 17%|█▋ | 134MB / 805MB, 25.8MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 70.2MB / 2.35GB  ...nts/optim_055000_rank1.pt: 0%| | 2.93MB / 2.49GB  ...nts/optim_055000_rank2.pt: 0%| | 2.96MB / 2.49GB  ...nts/optim_055000_rank3.pt: 2%|▏ | 56.2MB / 2.49GB  ...nts/optim_055000_rank4.pt: 0%| | 4.07MB / 2.49GB  ...nts/optim_055000_rank5.pt: 0%| | 2.93MB / 2.49GB  ...nts/optim_055000_rank7.pt: 1%| | 28.7MB / 2.49GB  ...nts/optim_055000_rank0.pt: 0%| | 7.95MB / 2.49GB  Processing Files (0 / 8) : 1%| | 176MB / 22.2GB, 32.6MB/s New Data Upload : 19%|█▉ | 155MB / 805MB, 28.8MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 70.2MB / 2.35GB  ...nts/optim_055000_rank1.pt: 0%| | 3.45MB / 2.49GB  ...nts/optim_055000_rank2.pt: 0%| | 3.47MB / 2.49GB  ...nts/optim_055000_rank3.pt: 2%|▏ | 60.9MB / 2.49GB  ...nts/optim_055000_rank4.pt: 0%| | 4.59MB / 2.49GB  ...nts/optim_055000_rank5.pt: 0%| | 3.45MB / 2.49GB  ...nts/optim_055000_rank7.pt: 1%|▏ | 33.9MB / 2.49GB  ...nts/optim_055000_rank0.pt: 1%| | 14.6MB / 2.49GB  Processing Files (0 / 8) : 1%| | 195MB / 22.2GB, 34.7MB/s New Data Upload : 20%|█▉ | 171MB / 872MB, 30.6MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 70.2MB / 2.35GB  ...nts/optim_055000_rank1.pt: 0%| | 4.01MB / 2.49GB  ...nts/optim_055000_rank2.pt: 0%| | 4.07MB / 2.49GB  ...nts/optim_055000_rank3.pt: 3%|▎ | 66.8MB / 2.49GB  ...nts/optim_055000_rank4.pt: 0%| | 5.72MB / 2.49GB  ...nts/optim_055000_rank5.pt: 0%| | 4.63MB / 2.49GB  ...nts/optim_055000_rank7.pt: 2%|▏ | 42.1MB / 2.49GB  ...nts/optim_055000_rank0.pt: 1%| | 20.8MB / 2.49GB  Processing Files (0 / 8) : 1%| | 218MB / 22.2GB, 37.6MB/s New Data Upload : 22%|██▏ | 192MB / 872MB, 33.1MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 70.2MB / 2.35GB  ...nts/optim_055000_rank1.pt: 0%| | 4.38MB / 2.49GB  ...nts/optim_055000_rank2.pt: 0%| | 4.51MB / 2.49GB  ...nts/optim_055000_rank3.pt: 3%|▎ | 66.9MB / 2.49GB  ...nts/optim_055000_rank4.pt: 0%| | 8.94MB / 2.49GB  ...nts/optim_055000_rank5.pt: 0%| | 5.07MB / 2.49GB  ...nts/optim_055000_rank7.pt: 2%|▏ | 50.7MB / 2.49GB  ...nts/optim_055000_rank0.pt: 1%| | 27.3MB / 2.49GB  Processing Files (0 / 8) : 1%| | 238MB / 22.2GB, 39.7MB/s New Data Upload : 24%|██▍ | 209MB / 872MB, 34.9MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 70.2MB / 2.35GB  ...nts/optim_055000_rank1.pt: 0%| | 4.69MB / 2.49GB  ...nts/optim_055000_rank2.pt: 0%| | 4.89MB / 2.49GB  ...nts/optim_055000_rank3.pt: 3%|▎ | 66.9MB / 2.49GB  ...nts/optim_055000_rank4.pt: 1%| | 12.7MB / 2.49GB  ...nts/optim_055000_rank5.pt: 0%| | 5.45MB / 2.49GB  ...nts/optim_055000_rank7.pt: 2%|▏ | 60.5MB / 2.49GB  ...nts/optim_055000_rank0.pt: 1%|▏ | 32.0MB / 2.49GB  Processing Files (0 / 8) : 1%| | 257MB / 22.2GB, 41.5MB/s New Data Upload : 24%|██▍ | 227MB / 939MB, 36.6MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 70.2MB / 2.35GB  ...nts/optim_055000_rank1.pt: 0%| | 5.55MB / 2.49GB  ...nts/optim_055000_rank2.pt: 0%| | 5.63MB / 2.49GB  ...nts/optim_055000_rank3.pt: 3%|▎ | 67.2MB / 2.49GB  ...nts/optim_055000_rank4.pt: 1%| | 21.7MB / 2.49GB  ...nts/optim_055000_rank5.pt: 0%| | 7.30MB / 2.49GB  ...nts/optim_055000_rank7.pt: 3%|▎ | 69.4MB / 2.49GB  ...nts/optim_055000_rank0.pt: 2%|▏ | 38.0MB / 2.49GB  Processing Files (0 / 8) : 1%|▏ | 285MB / 22.2GB, 44.5MB/s New Data Upload : 27%|██▋ | 251MB / 939MB, 39.2MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 70.2MB / 2.35GB  ...nts/optim_055000_rank1.pt: 0%| | 6.84MB / 2.49GB  ...nts/optim_055000_rank2.pt: 0%| | 10.5MB / 2.49GB  ...nts/optim_055000_rank3.pt: 3%|▎ | 68.1MB / 2.49GB  ...nts/optim_055000_rank4.pt: 1%| | 26.3MB / 2.49GB  ...nts/optim_055000_rank5.pt: 1%| | 12.7MB / 2.49GB  ...nts/optim_055000_rank7.pt: 3%|▎ | 73.0MB / 2.49GB  ...nts/optim_055000_rank0.pt: 2%|▏ | 45.4MB / 2.49GB  Processing Files (0 / 8) : 1%|▏ | 313MB / 22.2GB, 47.4MB/s New Data Upload : 29%|██▉ | 275MB / 939MB, 41.6MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 70.2MB / 2.35GB  ...nts/optim_055000_rank1.pt: 0%| | 8.07MB / 2.49GB  ...nts/optim_055000_rank2.pt: 1%| | 13.7MB / 2.49GB  ...nts/optim_055000_rank3.pt: 3%|▎ | 68.6MB / 2.49GB  ...nts/optim_055000_rank4.pt: 1%|▏ | 35.3MB / 2.49GB  ...nts/optim_055000_rank5.pt: 1%| | 18.7MB / 2.49GB  ...nts/optim_055000_rank7.pt: 3%|▎ | 73.6MB / 2.49GB  ...nts/optim_055000_rank0.pt: 2%|▏ | 48.6MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 337MB / 22.2GB, 49.5MB/s New Data Upload : 31%|███▏ | 295MB / 939MB, 43.3MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 71.3MB / 2.35GB  ...nts/optim_055000_rank1.pt: 0%| | 10.1MB / 2.49GB  ...nts/optim_055000_rank2.pt: 1%| | 22.6MB / 2.49GB  ...nts/optim_055000_rank3.pt: 3%|▎ | 69.4MB / 2.49GB  ...nts/optim_055000_rank4.pt: 2%|▏ | 40.6MB / 2.49GB  ...nts/optim_055000_rank5.pt: 1%| | 24.2MB / 2.49GB  ...nts/optim_055000_rank7.pt: 3%|▎ | 74.5MB / 2.49GB  ...nts/optim_055000_rank0.pt: 2%|▏ | 54.4MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 367MB / 22.2GB, 52.4MB/s New Data Upload : 30%|██▉ | 320MB / 1.07GB, 45.7MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 71.3MB / 2.35GB  ...nts/optim_055000_rank1.pt: 0%| | 12.2MB / 2.49GB  ...nts/optim_055000_rank2.pt: 1%|▏ | 32.6MB / 2.49GB  ...nts/optim_055000_rank3.pt: 3%|▎ | 70.0MB / 2.49GB  ...nts/optim_055000_rank4.pt: 2%|▏ | 48.3MB / 2.49GB  ...nts/optim_055000_rank5.pt: 1%| | 25.2MB / 2.49GB  ...nts/optim_055000_rank7.pt: 3%|▎ | 75.4MB / 2.49GB  ...nts/optim_055000_rank0.pt: 2%|▏ | 60.2MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 395MB / 22.2GB, 54.9MB/s New Data Upload : 32%|███▏ | 343MB / 1.07GB, 47.6MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 73.0MB / 2.35GB  ...nts/optim_055000_rank1.pt: 1%| | 16.8MB / 2.49GB  ...nts/optim_055000_rank2.pt: 2%|▏ | 43.4MB / 2.49GB  ...nts/optim_055000_rank3.pt: 3%|▎ | 75.1MB / 2.49GB  ...nts/optim_055000_rank4.pt: 2%|▏ | 58.3MB / 2.49GB  ...nts/optim_055000_rank5.pt: 1%| | 28.8MB / 2.49GB  ...nts/optim_055000_rank7.pt: 3%|▎ | 77.4MB / 2.49GB  ...nts/optim_055000_rank0.pt: 3%|▎ | 66.8MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 440MB / 22.2GB, 59.4MB/s New Data Upload : 33%|███▎ | 377MB / 1.14GB, 50.9MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 74.7MB / 2.35GB  ...nts/optim_055000_rank1.pt: 1%| | 20.1MB / 2.49GB  ...nts/optim_055000_rank2.pt: 2%|▏ | 44.7MB / 2.49GB  ...nts/optim_055000_rank3.pt: 3%|▎ | 75.6MB / 2.49GB  ...nts/optim_055000_rank4.pt: 3%|▎ | 63.8MB / 2.49GB  ...nts/optim_055000_rank5.pt: 1%|▏ | 33.4MB / 2.49GB  ...nts/optim_055000_rank7.pt: 3%|▎ | 78.5MB / 2.49GB  ...nts/optim_055000_rank0.pt: 3%|▎ | 72.3MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 463MB / 22.2GB, 60.9MB/s New Data Upload : 35%|███▍ | 395MB / 1.14GB, 51.9MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 76.5MB / 2.35GB  ...nts/optim_055000_rank1.pt: 1%| | 21.2MB / 2.49GB  ...nts/optim_055000_rank2.pt: 2%|▏ | 53.4MB / 2.49GB  ...nts/optim_055000_rank3.pt: 3%|▎ | 79.4MB / 2.49GB  ...nts/optim_055000_rank4.pt: 3%|▎ | 70.4MB / 2.49GB  ...nts/optim_055000_rank5.pt: 2%|▏ | 41.5MB / 2.49GB  ...nts/optim_055000_rank7.pt: 3%|▎ | 79.6MB / 2.49GB  ...nts/optim_055000_rank0.pt: 3%|▎ | 74.9MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 497MB / 22.2GB, 63.7MB/s New Data Upload : 37%|███▋ | 422MB / 1.14GB, 54.1MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 76.5MB / 2.35GB  ...nts/optim_055000_rank1.pt: 1%| | 25.2MB / 2.49GB  ...nts/optim_055000_rank2.pt: 3%|▎ | 64.9MB / 2.49GB  ...nts/optim_055000_rank3.pt: 3%|▎ | 84.1MB / 2.49GB  ...nts/optim_055000_rank4.pt: 3%|▎ | 75.4MB / 2.49GB  ...nts/optim_055000_rank5.pt: 2%|▏ | 50.8MB / 2.49GB  ...nts/optim_055000_rank7.pt: 3%|▎ | 81.8MB / 2.49GB  ...nts/optim_055000_rank0.pt: 3%|▎ | 77.2MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 536MB / 22.2GB, 67.0MB/s New Data Upload : 39%|███▉ | 448MB / 1.14GB, 56.0MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 78.2MB / 2.35GB  ...nts/optim_055000_rank1.pt: 1%| | 30.1MB / 2.49GB  ...nts/optim_055000_rank2.pt: 3%|▎ | 77.1MB / 2.49GB  ...nts/optim_055000_rank3.pt: 3%|▎ | 86.9MB / 2.49GB  ...nts/optim_055000_rank4.pt: 3%|▎ | 81.3MB / 2.49GB  ...nts/optim_055000_rank5.pt: 2%|▏ | 60.5MB / 2.49GB  ...nts/optim_055000_rank7.pt: 3%|▎ | 85.1MB / 2.49GB  ...nts/optim_055000_rank0.pt: 3%|▎ | 80.4MB / 2.49GB  Processing Files (0 / 8) : 3%|▎ | 580MB / 22.2GB, 70.7MB/s New Data Upload : 42%|████▏ | 474MB / 1.14GB, 57.8MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 79.9MB / 2.35GB  ...nts/optim_055000_rank1.pt: 2%|▏ | 37.5MB / 2.49GB  ...nts/optim_055000_rank2.pt: 4%|▎ | 89.0MB / 2.49GB  ...nts/optim_055000_rank3.pt: 4%|▎ | 92.8MB / 2.49GB  ...nts/optim_055000_rank4.pt: 3%|▎ | 87.0MB / 2.49GB  ...nts/optim_055000_rank5.pt: 3%|▎ | 72.1MB / 2.49GB  ...nts/optim_055000_rank7.pt: 4%|▎ | 90.9MB / 2.49GB  ...nts/optim_055000_rank0.pt: 3%|▎ | 86.4MB / 2.49GB  Processing Files (0 / 8) : 3%|▎ | 636MB / 22.2GB, 75.7MB/s New Data Upload : 44%|████▍ | 499MB / 1.14GB, 59.4MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 81.0MB / 2.35GB  ...nts/optim_055000_rank1.pt: 2%|▏ | 52.1MB / 2.49GB  ...nts/optim_055000_rank2.pt: 4%|▍ | 102MB / 2.49GB  ...nts/optim_055000_rank3.pt: 4%|▍ | 101MB / 2.49GB  ...nts/optim_055000_rank4.pt: 4%|▍ | 99.7MB / 2.49GB  ...nts/optim_055000_rank5.pt: 4%|▎ | 92.1MB / 2.49GB  ...nts/optim_055000_rank7.pt: 4%|▍ | 104MB / 2.49GB  ...nts/optim_055000_rank0.pt: 4%|▍ | 100MB / 2.49GB  Processing Files (0 / 8) : 3%|▎ | 732MB / 22.2GB, 85.1MB/s New Data Upload : 46%|████▌ | 525MB / 1.14GB, 61.1MB/s  ...eckpoints/model_055000.pt: 3%|▎ | 81.6MB / 2.35GB  ...nts/optim_055000_rank1.pt: 3%|▎ | 67.3MB / 2.49GB  ...nts/optim_055000_rank2.pt: 5%|▍ | 115MB / 2.49GB  ...nts/optim_055000_rank3.pt: 4%|▍ | 108MB / 2.49GB  ...nts/optim_055000_rank4.pt: 5%|▍ | 113MB / 2.49GB  ...nts/optim_055000_rank5.pt: 4%|▍ | 110MB / 2.49GB  ...nts/optim_055000_rank7.pt: 5%|▍ | 118MB / 2.49GB  ...nts/optim_055000_rank0.pt: 5%|▍ | 114MB / 2.49GB  Processing Files (0 / 8) : 4%|▎ | 827MB / 22.2GB, 94.0MB/s New Data Upload : 41%|████ | 548MB / 1.34GB, 62.3MB/s  ...eckpoints/model_055000.pt: 4%|▎ | 84.4MB / 2.35GB  ...nts/optim_055000_rank1.pt: 3%|▎ | 75.9MB / 2.49GB  ...nts/optim_055000_rank2.pt: 5%|▍ | 122MB / 2.49GB  ...nts/optim_055000_rank3.pt: 5%|▍ | 116MB / 2.49GB  ...nts/optim_055000_rank4.pt: 5%|▍ | 120MB / 2.49GB  ...nts/optim_055000_rank5.pt: 5%|▍ | 123MB / 2.49GB  ...nts/optim_055000_rank7.pt: 5%|▌ | 125MB / 2.49GB  ...nts/optim_055000_rank0.pt: 5%|▍ | 121MB / 2.49GB  Processing Files (0 / 8) : 4%|▍ | 888MB / 22.2GB, 98.7MB/s New Data Upload : 39%|███▊ | 568MB / 1.48GB, 63.1MB/s  ...eckpoints/model_055000.pt: 4%|▎ | 84.4MB / 2.35GB  ...nts/optim_055000_rank1.pt: 4%|▍ | 101MB / 2.49GB  ...nts/optim_055000_rank2.pt: 6%|▌ | 145MB / 2.49GB  ...nts/optim_055000_rank3.pt: 5%|▌ | 135MB / 2.49GB  ...nts/optim_055000_rank4.pt: 6%|▌ | 144MB / 2.49GB  ...nts/optim_055000_rank5.pt: 6%|▌ | 148MB / 2.49GB  ...nts/optim_055000_rank7.pt: 6%|▌ | 148MB / 2.49GB  ...nts/optim_055000_rank0.pt: 6%|▌ | 145MB / 2.49GB  Processing Files (0 / 8) : 5%|▍ | 1.05GB / 22.2GB, 114MB/s New Data Upload : 39%|███▉ | 598MB / 1.54GB, 65.0MB/s  ...eckpoints/model_055000.pt: 4%|▎ | 84.4MB / 2.35GB  ...nts/optim_055000_rank1.pt: 5%|▌ | 126MB / 2.49GB  ...nts/optim_055000_rank2.pt: 7%|▋ | 171MB / 2.49GB  ...nts/optim_055000_rank3.pt: 6%|▋ | 156MB / 2.49GB  ...nts/optim_055000_rank4.pt: 7%|▋ | 169MB / 2.49GB  ...nts/optim_055000_rank5.pt: 7%|▋ | 173MB / 2.49GB  ...nts/optim_055000_rank7.pt: 7%|▋ | 173MB / 2.49GB  ...nts/optim_055000_rank0.pt: 7%|▋ | 171MB / 2.49GB  Processing Files (0 / 8) : 6%|▌ | 1.22GB / 22.2GB, 130MB/s New Data Upload : 41%|████ | 629MB / 1.54GB, 66.9MB/s  ...eckpoints/model_055000.pt: 4%|▎ | 86.7MB / 2.35GB  ...nts/optim_055000_rank1.pt: 6%|▌ | 151MB / 2.49GB  ...nts/optim_055000_rank2.pt: 8%|▊ | 194MB / 2.49GB  ...nts/optim_055000_rank3.pt: 7%|▋ | 180MB / 2.49GB  ...nts/optim_055000_rank4.pt: 8%|▊ | 193MB / 2.49GB  ...nts/optim_055000_rank5.pt: 8%|▊ | 199MB / 2.49GB  ...nts/optim_055000_rank7.pt: 8%|▊ | 197MB / 2.49GB  ...nts/optim_055000_rank0.pt: 8%|▊ | 194MB / 2.49GB  Processing Files (0 / 8) : 6%|▋ | 1.39GB / 22.2GB, 145MB/s New Data Upload : 43%|████▎ | 663MB / 1.54GB, 69.0MB/s  ...eckpoints/model_055000.pt: 4%|▎ | 87.3MB / 2.35GB  ...nts/optim_055000_rank1.pt: 7%|▋ | 169MB / 2.49GB  ...nts/optim_055000_rank2.pt: 8%|▊ | 211MB / 2.49GB  ...nts/optim_055000_rank3.pt: 8%|▊ | 195MB / 2.49GB  ...nts/optim_055000_rank4.pt: 8%|▊ | 209MB / 2.49GB  ...nts/optim_055000_rank5.pt: 9%|▊ | 215MB / 2.49GB  ...nts/optim_055000_rank7.pt: 9%|▊ | 213MB / 2.49GB  ...nts/optim_055000_rank0.pt: 8%|▊ | 211MB / 2.49GB  Processing Files (0 / 8) : 7%|▋ | 1.51GB / 22.2GB, 154MB/s New Data Upload : 41%|████ | 684MB / 1.68GB, 69.8MB/s  ...eckpoints/model_055000.pt: 4%|▍ | 89.0MB / 2.35GB  ...nts/optim_055000_rank1.pt: 8%|▊ | 197MB / 2.49GB  ...nts/optim_055000_rank2.pt: 10%|▉ | 238MB / 2.49GB  ...nts/optim_055000_rank3.pt: 9%|▉ | 221MB / 2.49GB  ...nts/optim_055000_rank4.pt: 10%|▉ | 238MB / 2.49GB  ...nts/optim_055000_rank5.pt: 10%|▉ | 244MB / 2.49GB  ...nts/optim_055000_rank7.pt: 10%|▉ | 241MB / 2.49GB  ...nts/optim_055000_rank0.pt: 10%|▉ | 237MB / 2.49GB  Processing Files (0 / 8) : 8%|▊ | 1.71GB / 22.2GB, 171MB/s New Data Upload : 41%|████▏ | 723MB / 1.74GB, 72.3MB/s  ...eckpoints/model_055000.pt: 4%|▍ | 89.0MB / 2.35GB  ...nts/optim_055000_rank1.pt: 9%|▉ | 224MB / 2.49GB  ...nts/optim_055000_rank2.pt: 11%|█ | 263MB / 2.49GB  ...nts/optim_055000_rank3.pt: 10%|▉ | 246MB / 2.49GB  ...nts/optim_055000_rank4.pt: 11%|█ | 264MB / 2.49GB  ...nts/optim_055000_rank5.pt: 11%|█ | 272MB / 2.49GB  ...nts/optim_055000_rank7.pt: 11%|█ | 266MB / 2.49GB  ...nts/optim_055000_rank0.pt: 11%|█ | 263MB / 2.49GB  Processing Files (0 / 8) : 8%|▊ | 1.89GB / 22.2GB, 185MB/s New Data Upload : 43%|████▎ | 757MB / 1.74GB, 74.2MB/s  ...eckpoints/model_055000.pt: 4%|▍ | 89.6MB / 2.35GB  ...nts/optim_055000_rank1.pt: 10%|▉ | 247MB / 2.49GB  ...nts/optim_055000_rank2.pt: 11%|█▏ | 284MB / 2.49GB  ...nts/optim_055000_rank3.pt: 11%|█ | 269MB / 2.49GB  ...nts/optim_055000_rank4.pt: 11%|█▏ | 285MB / 2.49GB  ...nts/optim_055000_rank5.pt: 12%|█▏ | 294MB / 2.49GB  ...nts/optim_055000_rank7.pt: 12%|█▏ | 287MB / 2.49GB  ...nts/optim_055000_rank0.pt: 11%|█▏ | 284MB / 2.49GB  Processing Files (0 / 8) : 9%|▉ | 2.04GB / 22.2GB, 200MB/s New Data Upload : 45%|████▍ | 785MB / 1.74GB, 76.9MB/s  ...eckpoints/model_055000.pt: 4%|▍ | 90.7MB / 2.35GB  ...nts/optim_055000_rank1.pt: 11%|█ | 266MB / 2.49GB  ...nts/optim_055000_rank2.pt: 12%|█▏ | 302MB / 2.49GB  ...nts/optim_055000_rank3.pt: 12%|█▏ | 294MB / 2.49GB  ...nts/optim_055000_rank4.pt: 12%|█▏ | 304MB / 2.49GB  ...nts/optim_055000_rank5.pt: 13%|█▎ | 315MB / 2.49GB  ...nts/optim_055000_rank7.pt: 12%|█▏ | 304MB / 2.49GB  ...nts/optim_055000_rank0.pt: 12%|█▏ | 301MB / 2.49GB  Processing Files (0 / 8) : 10%|▉ | 2.18GB / 22.2GB, 213MB/s New Data Upload : 45%|████▍ | 813MB / 1.81GB, 79.8MB/s  ...eckpoints/model_055000.pt: 4%|▍ | 91.8MB / 2.35GB  ...nts/optim_055000_rank1.pt: 12%|█▏ | 294MB / 2.49GB  ...nts/optim_055000_rank2.pt: 13%|█▎ | 329MB / 2.49GB  ...nts/optim_055000_rank3.pt: 13%|█▎ | 323MB / 2.49GB  ...nts/optim_055000_rank4.pt: 13%|█▎ | 331MB / 2.49GB  ...nts/optim_055000_rank5.pt: 14%|█▍ | 342MB / 2.49GB  ...nts/optim_055000_rank7.pt: 13%|█▎ | 332MB / 2.49GB  ...nts/optim_055000_rank0.pt: 13%|█▎ | 327MB / 2.49GB  Processing Files (0 / 8) : 11%|█ | 2.37GB / 22.2GB, 232MB/s New Data Upload : 47%|████▋ | 850MB / 1.81GB, 83.3MB/s  ...eckpoints/model_055000.pt: 4%|▍ | 92.9MB / 2.35GB  ...nts/optim_055000_rank1.pt: 13%|█▎ | 321MB / 2.49GB  ...nts/optim_055000_rank2.pt: 14%|█▍ | 354MB / 2.49GB  ...nts/optim_055000_rank3.pt: 14%|█▍ | 350MB / 2.49GB  ...nts/optim_055000_rank4.pt: 14%|█▍ | 359MB / 2.49GB  ...nts/optim_055000_rank5.pt: 15%|█▍ | 369MB / 2.49GB  ...nts/optim_055000_rank7.pt: 15%|█▍ | 361MB / 2.49GB  ...nts/optim_055000_rank0.pt: 14%|█▍ | 352MB / 2.49GB  Processing Files (0 / 8) : 12%|█▏ | 2.56GB / 22.2GB, 250MB/s New Data Upload : 46%|████▌ | 888MB / 1.94GB, 87.0MB/s  ...eckpoints/model_055000.pt: 4%|▍ | 95.7MB / 2.35GB  ...nts/optim_055000_rank1.pt: 14%|█▍ | 350MB / 2.49GB  ...nts/optim_055000_rank2.pt: 15%|█▌ | 380MB / 2.49GB  ...nts/optim_055000_rank3.pt: 15%|█▌ | 379MB / 2.49GB  ...nts/optim_055000_rank4.pt: 16%|█▌ | 386MB / 2.49GB  ...nts/optim_055000_rank5.pt: 16%|█▌ | 399MB / 2.49GB  ...nts/optim_055000_rank7.pt: 16%|█▌ | 386MB / 2.49GB  ...nts/optim_055000_rank0.pt: 15%|█▌ | 379MB / 2.49GB  Processing Files (0 / 8) : 12%|█▏ | 2.76GB / 22.2GB, 270MB/s New Data Upload : 48%|████▊ | 930MB / 1.94GB, 91.1MB/s  ...eckpoints/model_055000.pt: 4%|▍ | 96.3MB / 2.35GB  ...nts/optim_055000_rank1.pt: 15%|█▌ | 379MB / 2.49GB  ...nts/optim_055000_rank2.pt: 16%|█▋ | 407MB / 2.49GB  ...nts/optim_055000_rank3.pt: 16%|█▋ | 408MB / 2.49GB  ...nts/optim_055000_rank4.pt: 17%|█▋ | 414MB / 2.49GB  ...nts/optim_055000_rank5.pt: 17%|█▋ | 427MB / 2.49GB  ...nts/optim_055000_rank7.pt: 17%|█▋ | 413MB / 2.49GB  ...nts/optim_055000_rank0.pt: 16%|█▋ | 407MB / 2.49GB  Processing Files (0 / 8) : 13%|█▎ | 2.95GB / 22.2GB, 289MB/s New Data Upload : 50%|████▉ | 971MB / 1.94GB, 95.1MB/s  ...eckpoints/model_055000.pt: 4%|▍ | 98.6MB / 2.35GB  ...nts/optim_055000_rank1.pt: 17%|█▋ | 417MB / 2.49GB  ...nts/optim_055000_rank2.pt: 18%|█▊ | 443MB / 2.49GB  ...nts/optim_055000_rank3.pt: 18%|█▊ | 443MB / 2.49GB  ...nts/optim_055000_rank4.pt: 18%|█▊ | 448MB / 2.49GB  ...nts/optim_055000_rank5.pt: 19%|█▊ | 461MB / 2.49GB  ...nts/optim_055000_rank7.pt: 18%|█▊ | 447MB / 2.49GB  ...nts/optim_055000_rank0.pt: 18%|█▊ | 442MB / 2.49GB  Processing Files (0 / 8) : 14%|█▍ | 3.20GB / 22.2GB, 313MB/s New Data Upload : 51%|█████ | 1.02GB / 2.01GB, 99.7MB/s  ...eckpoints/model_055000.pt: 4%|▍ | 99.1MB / 2.35GB  ...nts/optim_055000_rank1.pt: 18%|█▊ | 440MB / 2.49GB  ...nts/optim_055000_rank2.pt: 19%|█▊ | 463MB / 2.49GB  ...nts/optim_055000_rank3.pt: 19%|█▊ | 465MB / 2.49GB  ...nts/optim_055000_rank4.pt: 19%|█▉ | 469MB / 2.49GB  ...nts/optim_055000_rank5.pt: 19%|█▉ | 484MB / 2.49GB  ...nts/optim_055000_rank7.pt: 19%|█▉ | 469MB / 2.49GB  ...nts/optim_055000_rank0.pt: 19%|█▊ | 464MB / 2.49GB  Processing Files (0 / 8) : 15%|█▌ | 3.35GB / 22.2GB, 328MB/s New Data Upload : 51%|█████ | 1.06GB / 2.08GB, 103MB/s  ...eckpoints/model_055000.pt: 4%|▍ | 99.7MB / 2.35GB  ...nts/optim_055000_rank1.pt: 19%|█▊ | 465MB / 2.49GB  ...nts/optim_055000_rank2.pt: 20%|█▉ | 487MB / 2.49GB  ...nts/optim_055000_rank3.pt: 20%|█▉ | 490MB / 2.49GB  ...nts/optim_055000_rank4.pt: 20%|█▉ | 493MB / 2.49GB  ...nts/optim_055000_rank5.pt: 20%|██ | 508MB / 2.49GB  ...nts/optim_055000_rank7.pt: 20%|█▉ | 493MB / 2.49GB  ...nts/optim_055000_rank0.pt: 20%|█▉ | 488MB / 2.49GB  Processing Files (0 / 8) : 16%|█▌ | 3.52GB / 22.2GB, 344MB/s New Data Upload : 51%|█████ | 1.09GB / 2.15GB, 106MB/s  ...eckpoints/model_055000.pt: 4%|▍ | 103MB / 2.35GB  ...nts/optim_055000_rank1.pt: 19%|█▉ | 485MB / 2.49GB  ...nts/optim_055000_rank2.pt: 20%|██ | 506MB / 2.49GB  ...nts/optim_055000_rank3.pt: 20%|██ | 509MB / 2.49GB  ...nts/optim_055000_rank4.pt: 21%|██ | 512MB / 2.49GB  ...nts/optim_055000_rank5.pt: 21%|██ | 526MB / 2.49GB  ...nts/optim_055000_rank7.pt: 21%|██ | 512MB / 2.49GB  ...nts/optim_055000_rank0.pt: 20%|██ | 504MB / 2.49GB  Processing Files (0 / 8) : 16%|█▋ | 3.66GB / 22.2GB, 356MB/s New Data Upload : 53%|█████▎ | 1.13GB / 2.15GB, 109MB/s  ...eckpoints/model_055000.pt: 4%|▍ | 103MB / 2.35GB  ...nts/optim_055000_rank1.pt: 20%|██ | 503MB / 2.49GB  ...nts/optim_055000_rank2.pt: 21%|██ | 521MB / 2.49GB  ...nts/optim_055000_rank3.pt: 21%|██ | 526MB / 2.49GB  ...nts/optim_055000_rank4.pt: 21%|██▏ | 529MB / 2.49GB  ...nts/optim_055000_rank5.pt: 22%|██▏ | 541MB / 2.49GB  ...nts/optim_055000_rank7.pt: 21%|██▏ | 529MB / 2.49GB  ...nts/optim_055000_rank0.pt: 21%|██ | 520MB / 2.49GB  Processing Files (0 / 8) : 17%|█▋ | 3.77GB / 22.2GB, 367MB/s New Data Upload : 54%|█████▍ | 1.16GB / 2.15GB, 111MB/s  ...eckpoints/model_055000.pt: 5%|▍ | 106MB / 2.35GB  ...nts/optim_055000_rank1.pt: 21%|██ | 526MB / 2.49GB  ...nts/optim_055000_rank2.pt: 22%|██▏ | 540MB / 2.49GB  ...nts/optim_055000_rank3.pt: 22%|██▏ | 548MB / 2.49GB  ...nts/optim_055000_rank4.pt: 22%|██▏ | 548MB / 2.49GB  ...nts/optim_055000_rank5.pt: 22%|██▏ | 558MB / 2.49GB  ...nts/optim_055000_rank7.pt: 22%|██▏ | 547MB / 2.49GB  ...nts/optim_055000_rank0.pt: 22%|██▏ | 540MB / 2.49GB  Processing Files (0 / 8) : 18%|█▊ | 3.91GB / 22.2GB, 379MB/s New Data Upload : 54%|█████▍ | 1.20GB / 2.21GB, 113MB/s  ...eckpoints/model_055000.pt: 5%|▍ | 106MB / 2.35GB  ...nts/optim_055000_rank1.pt: 22%|██▏ | 554MB / 2.49GB  ...nts/optim_055000_rank2.pt: 23%|██▎ | 564MB / 2.49GB  ...nts/optim_055000_rank3.pt: 23%|██▎ | 569MB / 2.49GB  ...nts/optim_055000_rank4.pt: 23%|██▎ | 571MB / 2.49GB  ...nts/optim_055000_rank5.pt: 23%|██▎ | 578MB / 2.49GB  ...nts/optim_055000_rank7.pt: 23%|██▎ | 571MB / 2.49GB  ...nts/optim_055000_rank0.pt: 23%|██▎ | 562MB / 2.49GB  Processing Files (0 / 8) : 18%|█▊ | 4.07GB / 22.2GB, 394MB/s New Data Upload : 56%|█████▌ | 1.24GB / 2.21GB, 117MB/s  ...eckpoints/model_055000.pt: 5%|▍ | 111MB / 2.35GB  ...nts/optim_055000_rank1.pt: 23%|██▎ | 580MB / 2.49GB  ...nts/optim_055000_rank2.pt: 24%|██▎ | 585MB / 2.49GB  ...nts/optim_055000_rank3.pt: 24%|██▍ | 592MB / 2.49GB  ...nts/optim_055000_rank4.pt: 24%|██▍ | 593MB / 2.49GB  ...nts/optim_055000_rank5.pt: 24%|██▍ | 597MB / 2.49GB  ...nts/optim_055000_rank7.pt: 24%|██▍ | 594MB / 2.49GB  ...nts/optim_055000_rank0.pt: 23%|██▎ | 583MB / 2.49GB  Processing Files (0 / 8) : 19%|█▉ | 4.24GB / 22.2GB, 410MB/s New Data Upload : 58%|█████▊ | 1.28GB / 2.21GB, 120MB/s  ...eckpoints/model_055000.pt: 5%|▍ | 113MB / 2.35GB  ...nts/optim_055000_rank1.pt: 24%|██▍ | 604MB / 2.49GB  ...nts/optim_055000_rank2.pt: 25%|██▍ | 611MB / 2.49GB  ...nts/optim_055000_rank3.pt: 25%|██▍ | 617MB / 2.49GB  ...nts/optim_055000_rank4.pt: 25%|██▍ | 615MB / 2.49GB  ...nts/optim_055000_rank5.pt: 25%|██▍ | 619MB / 2.49GB  ...nts/optim_055000_rank7.pt: 25%|██▍ | 617MB / 2.49GB  ...nts/optim_055000_rank0.pt: 24%|██▍ | 606MB / 2.49GB  Processing Files (0 / 8) : 20%|█▉ | 4.40GB / 22.2GB, 425MB/s New Data Upload : 58%|█████▊ | 1.32GB / 2.28GB, 123MB/s  ...eckpoints/model_055000.pt: 5%|▌ | 119MB / 2.35GB  ...nts/optim_055000_rank1.pt: 26%|██▌ | 639MB / 2.49GB  ...nts/optim_055000_rank2.pt: 26%|██▌ | 642MB / 2.49GB  ...nts/optim_055000_rank3.pt: 26%|██▌ | 647MB / 2.49GB  ...nts/optim_055000_rank4.pt: 26%|██▌ | 647MB / 2.49GB  ...nts/optim_055000_rank5.pt: 26%|██▌ | 648MB / 2.49GB  ...nts/optim_055000_rank7.pt: 26%|██▌ | 650MB / 2.49GB  ...nts/optim_055000_rank0.pt: 26%|██▌ | 638MB / 2.49GB  Processing Files (0 / 8) : 21%|██ | 4.63GB / 22.2GB, 447MB/s New Data Upload : 58%|█████▊ | 1.37GB / 2.35GB, 128MB/s  ...eckpoints/model_055000.pt: 5%|▌ | 123MB / 2.35GB  ...nts/optim_055000_rank1.pt: 26%|██▋ | 657MB / 2.49GB  ...nts/optim_055000_rank2.pt: 27%|██▋ | 660MB / 2.49GB  ...nts/optim_055000_rank3.pt: 27%|██▋ | 666MB / 2.49GB  ...nts/optim_055000_rank4.pt: 27%|██▋ | 667MB / 2.49GB  ...nts/optim_055000_rank5.pt: 27%|██▋ | 666MB / 2.49GB  ...nts/optim_055000_rank7.pt: 27%|██▋ | 671MB / 2.49GB  ...nts/optim_055000_rank0.pt: 26%|██▋ | 655MB / 2.49GB  Processing Files (0 / 8) : 21%|██▏ | 4.76GB / 22.2GB, 460MB/s New Data Upload : 57%|█████▋ | 1.41GB / 2.48GB, 131MB/s  ...eckpoints/model_055000.pt: 6%|▌ | 131MB / 2.35GB  ...nts/optim_055000_rank1.pt: 27%|██▋ | 679MB / 2.49GB  ...nts/optim_055000_rank2.pt: 27%|██▋ | 678MB / 2.49GB  ...nts/optim_055000_rank3.pt: 27%|██▋ | 683MB / 2.49GB  ...nts/optim_055000_rank4.pt: 28%|██▊ | 684MB / 2.49GB  ...nts/optim_055000_rank5.pt: 27%|██▋ | 680MB / 2.49GB  ...nts/optim_055000_rank7.pt: 28%|██▊ | 688MB / 2.49GB  ...nts/optim_055000_rank0.pt: 27%|██▋ | 675MB / 2.49GB  Processing Files (0 / 8) : 22%|██▏ | 4.90GB / 22.2GB, 473MB/s New Data Upload : 57%|█████▋ | 1.44GB / 2.55GB, 135MB/s  ...eckpoints/model_055000.pt: 6%|▌ | 139MB / 2.35GB  ...nts/optim_055000_rank1.pt: 28%|██▊ | 699MB / 2.49GB  ...nts/optim_055000_rank2.pt: 28%|██▊ | 695MB / 2.49GB  ...nts/optim_055000_rank3.pt: 28%|██▊ | 699MB / 2.49GB  ...nts/optim_055000_rank4.pt: 28%|██▊ | 700MB / 2.49GB  ...nts/optim_055000_rank5.pt: 28%|██▊ | 698MB / 2.49GB  ...nts/optim_055000_rank7.pt: 28%|██▊ | 704MB / 2.49GB  ...nts/optim_055000_rank0.pt: 28%|██▊ | 693MB / 2.49GB  Processing Files (0 / 8) : 23%|██▎ | 5.03GB / 22.2GB, 486MB/s New Data Upload : 58%|█████▊ | 1.47GB / 2.55GB, 138MB/s  ...eckpoints/model_055000.pt: 6%|▌ | 141MB / 2.35GB  ...nts/optim_055000_rank1.pt: 29%|██▉ | 727MB / 2.49GB  ...nts/optim_055000_rank2.pt: 29%|██▉ | 723MB / 2.49GB  ...nts/optim_055000_rank3.pt: 29%|██▉ | 728MB / 2.49GB  ...nts/optim_055000_rank4.pt: 29%|██▉ | 729MB / 2.49GB  ...nts/optim_055000_rank5.pt: 29%|██▉ | 724MB / 2.49GB  ...nts/optim_055000_rank7.pt: 30%|██▉ | 734MB / 2.49GB  ...nts/optim_055000_rank0.pt: 29%|██▉ | 723MB / 2.49GB  Processing Files (0 / 8) : 24%|██▎ | 5.23GB / 22.2GB, 506MB/s New Data Upload : 58%|█████▊ | 1.52GB / 2.62GB, 142MB/s  ...eckpoints/model_055000.pt: 6%|▋ | 147MB / 2.35GB  ...nts/optim_055000_rank1.pt: 30%|███ | 753MB / 2.49GB  ...nts/optim_055000_rank2.pt: 30%|███ | 746MB / 2.49GB  ...nts/optim_055000_rank3.pt: 30%|███ | 754MB / 2.49GB  ...nts/optim_055000_rank4.pt: 30%|███ | 755MB / 2.49GB  ...nts/optim_055000_rank5.pt: 30%|███ | 749MB / 2.49GB  ...nts/optim_055000_rank7.pt: 31%|███ | 760MB / 2.49GB  ...nts/optim_055000_rank0.pt: 30%|███ | 750MB / 2.49GB  Processing Files (0 / 8) : 24%|██▍ | 5.41GB / 22.2GB, 523MB/s New Data Upload : 59%|█████▉ | 1.56GB / 2.62GB, 145MB/s  ...eckpoints/model_055000.pt: 6%|▋ | 147MB / 2.35GB  ...nts/optim_055000_rank1.pt: 31%|███ | 775MB / 2.49GB  ...nts/optim_055000_rank2.pt: 31%|███ | 766MB / 2.49GB  ...nts/optim_055000_rank3.pt: 31%|███ | 775MB / 2.49GB  ...nts/optim_055000_rank4.pt: 31%|███ | 775MB / 2.49GB  ...nts/optim_055000_rank5.pt: 31%|███ | 768MB / 2.49GB  ...nts/optim_055000_rank7.pt: 31%|███▏ | 780MB / 2.49GB  ...nts/optim_055000_rank0.pt: 31%|███ | 772MB / 2.49GB  Processing Files (0 / 8) : 25%|██▌ | 5.56GB / 22.2GB, 536MB/s New Data Upload : 61%|██████ | 1.58GB / 2.62GB, 147MB/s  ...eckpoints/model_055000.pt: 7%|▋ | 156MB / 2.35GB  ...nts/optim_055000_rank1.pt: 32%|███▏ | 797MB / 2.49GB  ...nts/optim_055000_rank2.pt: 32%|███▏ | 786MB / 2.49GB  ...nts/optim_055000_rank3.pt: 32%|███▏ | 794MB / 2.49GB  ...nts/optim_055000_rank4.pt: 32%|███▏ | 794MB / 2.49GB  ...nts/optim_055000_rank5.pt: 32%|███▏ | 788MB / 2.49GB  ...nts/optim_055000_rank7.pt: 32%|███▏ | 799MB / 2.49GB  ...nts/optim_055000_rank0.pt: 32%|███▏ | 791MB / 2.49GB  Processing Files (0 / 8) : 26%|██▌ | 5.70GB / 22.2GB, 549MB/s New Data Upload : 58%|█████▊ | 1.62GB / 2.82GB, 150MB/s  ...eckpoints/model_055000.pt: 7%|▋ | 160MB / 2.35GB  ...nts/optim_055000_rank1.pt: 33%|███▎ | 827MB / 2.49GB  ...nts/optim_055000_rank2.pt: 33%|███▎ | 816MB / 2.49GB  ...nts/optim_055000_rank3.pt: 33%|███▎ | 826MB / 2.49GB  ...nts/optim_055000_rank4.pt: 33%|███▎ | 827MB / 2.49GB  ...nts/optim_055000_rank5.pt: 33%|███▎ | 818MB / 2.49GB  ...nts/optim_055000_rank7.pt: 34%|███▎ | 833MB / 2.49GB  ...nts/optim_055000_rank0.pt: 33%|███▎ | 824MB / 2.49GB  Processing Files (0 / 8) : 27%|██▋ | 5.93GB / 22.2GB, 571MB/s New Data Upload : 58%|█████▊ | 1.67GB / 2.88GB, 154MB/s  ...eckpoints/model_055000.pt: 7%|▋ | 167MB / 2.35GB  ...nts/optim_055000_rank1.pt: 34%|███▍ | 855MB / 2.49GB  ...nts/optim_055000_rank2.pt: 34%|███▍ | 843MB / 2.49GB  ...nts/optim_055000_rank3.pt: 34%|███▍ | 855MB / 2.49GB  ...nts/optim_055000_rank4.pt: 34%|███▍ | 855MB / 2.49GB  ...nts/optim_055000_rank5.pt: 34%|███▍ | 845MB / 2.49GB  ...nts/optim_055000_rank7.pt: 35%|███▍ | 861MB / 2.49GB  ...nts/optim_055000_rank0.pt: 34%|███▍ | 853MB / 2.49GB  Processing Files (0 / 8) : 28%|██▊ | 6.13GB / 22.2GB, 589MB/s New Data Upload : 58%|█████▊ | 1.71GB / 2.95GB, 157MB/s  ...eckpoints/model_055000.pt: 7%|▋ | 170MB / 2.35GB  ...nts/optim_055000_rank1.pt: 35%|███▌ | 873MB / 2.49GB  ...nts/optim_055000_rank2.pt: 35%|███▍ | 863MB / 2.49GB  ...nts/optim_055000_rank3.pt: 35%|███▌ | 875MB / 2.49GB  ...nts/optim_055000_rank4.pt: 35%|███▌ | 875MB / 2.49GB  ...nts/optim_055000_rank5.pt: 35%|███▍ | 867MB / 2.49GB  ...nts/optim_055000_rank7.pt: 35%|███▌ | 882MB / 2.49GB  ...nts/optim_055000_rank0.pt: 35%|███▌ | 873MB / 2.49GB  Processing Files (0 / 8) : 28%|██▊ | 6.28GB / 22.2GB, 601MB/s New Data Upload : 59%|█████▉ | 1.74GB / 2.95GB, 158MB/s  ...eckpoints/model_055000.pt: 8%|▊ | 180MB / 2.35GB  ...nts/optim_055000_rank1.pt: 36%|███▌ | 896MB / 2.49GB  ...nts/optim_055000_rank2.pt: 36%|███▌ | 884MB / 2.49GB  ...nts/optim_055000_rank3.pt: 36%|███▌ | 899MB / 2.49GB  ...nts/optim_055000_rank4.pt: 36%|███▌ | 898MB / 2.49GB  ...nts/optim_055000_rank5.pt: 36%|███▌ | 891MB / 2.49GB  ...nts/optim_055000_rank7.pt: 36%|███▋ | 905MB / 2.49GB  ...nts/optim_055000_rank0.pt: 36%|███▌ | 897MB / 2.49GB  Processing Files (0 / 8) : 29%|██▉ | 6.45GB / 22.2GB, 615MB/s New Data Upload : 58%|█████▊ | 1.78GB / 3.08GB, 160MB/s  ...eckpoints/model_055000.pt: 8%|▊ | 182MB / 2.35GB  ...nts/optim_055000_rank1.pt: 37%|███▋ | 920MB / 2.49GB  ...nts/optim_055000_rank2.pt: 37%|███▋ | 910MB / 2.49GB  ...nts/optim_055000_rank3.pt: 37%|███▋ | 921MB / 2.49GB  ...nts/optim_055000_rank4.pt: 37%|███▋ | 921MB / 2.49GB  ...nts/optim_055000_rank5.pt: 37%|███▋ | 916MB / 2.49GB  ...nts/optim_055000_rank7.pt: 37%|███▋ | 928MB / 2.49GB  ...nts/optim_055000_rank0.pt: 37%|███▋ | 919MB / 2.49GB  Processing Files (0 / 8) : 30%|██▉ | 6.62GB / 22.2GB, 630MB/s New Data Upload : 58%|█████▊ | 1.82GB / 3.15GB, 162MB/s  ...eckpoints/model_055000.pt: 8%|▊ | 191MB / 2.35GB  ...nts/optim_055000_rank1.pt: 38%|███▊ | 944MB / 2.49GB  ...nts/optim_055000_rank2.pt: 37%|███▋ | 931MB / 2.49GB  ...nts/optim_055000_rank3.pt: 38%|███▊ | 946MB / 2.49GB  ...nts/optim_055000_rank4.pt: 38%|███▊ | 945MB / 2.49GB  ...nts/optim_055000_rank5.pt: 38%|███▊ | 942MB / 2.49GB  ...nts/optim_055000_rank7.pt: 38%|███▊ | 953MB / 2.49GB  ...nts/optim_055000_rank0.pt: 38%|███▊ | 944MB / 2.49GB  Processing Files (0 / 8) : 31%|███ | 6.80GB / 22.2GB, 645MB/s New Data Upload : 59%|█████▉ | 1.86GB / 3.15GB, 164MB/s  ...eckpoints/model_055000.pt: 9%|▊ | 201MB / 2.35GB  ...nts/optim_055000_rank1.pt: 39%|███▊ | 963MB / 2.49GB  ...nts/optim_055000_rank2.pt: 38%|███▊ | 951MB / 2.49GB  ...nts/optim_055000_rank3.pt: 39%|███▉ | 963MB / 2.49GB  ...nts/optim_055000_rank4.pt: 39%|███▊ | 963MB / 2.49GB  ...nts/optim_055000_rank5.pt: 39%|███▊ | 959MB / 2.49GB  ...nts/optim_055000_rank7.pt: 39%|███▉ | 970MB / 2.49GB  ...nts/optim_055000_rank0.pt: 39%|███▊ | 962MB / 2.49GB  Processing Files (0 / 8) : 31%|███ | 6.93GB / 22.2GB, 656MB/s New Data Upload : 58%|█████▊ | 1.89GB / 3.29GB, 165MB/s  ...eckpoints/model_055000.pt: 9%|▉ | 208MB / 2.35GB  ...nts/optim_055000_rank1.pt: 40%|███▉ | 985MB / 2.49GB  ...nts/optim_055000_rank2.pt: 39%|███▉ | 976MB / 2.49GB  ...nts/optim_055000_rank3.pt: 40%|███▉ | 985MB / 2.49GB  ...nts/optim_055000_rank4.pt: 40%|███▉ | 984MB / 2.49GB  ...nts/optim_055000_rank5.pt: 40%|███▉ | 982MB / 2.49GB  ...nts/optim_055000_rank7.pt: 40%|███▉ | 992MB / 2.49GB  ...nts/optim_055000_rank0.pt: 40%|███▉ | 982MB / 2.49GB  Processing Files (0 / 8) : 32%|███▏ | 7.09GB / 22.2GB, 670MB/s New Data Upload : 58%|█████▊ | 1.93GB / 3.35GB, 167MB/s  ...eckpoints/model_055000.pt: 9%|▉ | 216MB / 2.35GB  ...nts/optim_055000_rank1.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_055000_rank2.pt: 40%|████ | 1.00GB / 2.49GB  ...nts/optim_055000_rank3.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_055000_rank4.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_055000_rank5.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_055000_rank7.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_055000_rank0.pt: 41%|████ | 1.01GB / 2.49GB  Processing Files (0 / 8) : 33%|███▎ | 7.28GB / 22.2GB, 686MB/s New Data Upload : 59%|█████▉ | 1.97GB / 3.35GB, 169MB/s  ...eckpoints/model_055000.pt: 9%|▉ | 222MB / 2.35GB  ...nts/optim_055000_rank1.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_055000_rank2.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_055000_rank3.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_055000_rank4.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_055000_rank5.pt: 42%|████▏ | 1.03GB / 2.49GB  ...nts/optim_055000_rank7.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_055000_rank0.pt: 42%|████▏ | 1.03GB / 2.49GB  Processing Files (0 / 8) : 34%|███▎ | 7.48GB / 22.2GB, 702MB/s New Data Upload : 59%|█████▉ | 2.01GB / 3.42GB, 171MB/s  ...eckpoints/model_055000.pt: 10%|▉ | 232MB / 2.35GB  ...nts/optim_055000_rank1.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_055000_rank2.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_055000_rank3.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_055000_rank4.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_055000_rank5.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_055000_rank7.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_055000_rank0.pt: 42%|████▏ | 1.05GB / 2.49GB  Processing Files (0 / 8) : 34%|███▍ | 7.61GB / 22.2GB, 713MB/s New Data Upload : 59%|█████▉ | 2.05GB / 3.49GB, 172MB/s  ...eckpoints/model_055000.pt: 10%|█ | 241MB / 2.35GB  ...nts/optim_055000_rank1.pt: 44%|████▎ | 1.08GB / 2.49GB  ...nts/optim_055000_rank2.pt: 44%|████▎ | 1.08GB / 2.49GB  ...nts/optim_055000_rank3.pt: 43%|████▎ | 1.08GB / 2.49GB  ...nts/optim_055000_rank4.pt: 43%|████▎ | 1.08GB / 2.49GB  ...nts/optim_055000_rank5.pt: 43%|████▎ | 1.08GB / 2.49GB  ...nts/optim_055000_rank7.pt: 44%|████▎ | 1.08GB / 2.49GB  ...nts/optim_055000_rank0.pt: 43%|████▎ | 1.07GB / 2.49GB  Processing Files (0 / 8) : 35%|███▌ | 7.80GB / 22.2GB, 728MB/s New Data Upload : 60%|██████ | 2.09GB / 3.49GB, 174MB/s  ...eckpoints/model_055000.pt: 11%|█ | 253MB / 2.35GB  ...nts/optim_055000_rank1.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_055000_rank2.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_055000_rank3.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_055000_rank4.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_055000_rank5.pt: 44%|████▍ | 1.11GB / 2.49GB  ...nts/optim_055000_rank7.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_055000_rank0.pt: 44%|████▍ | 1.10GB / 2.49GB  Processing Files (0 / 8) : 36%|███▌ | 8.00GB / 22.2GB, 745MB/s New Data Upload : 62%|██████▏ | 2.14GB / 3.49GB, 177MB/s  ...eckpoints/model_055000.pt: 11%|█ | 261MB / 2.35GB  ...nts/optim_055000_rank1.pt: 46%|████▌ | 1.13GB / 2.49GB  ...nts/optim_055000_rank2.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_055000_rank3.pt: 45%|████▌ | 1.12GB / 2.49GB  ...nts/optim_055000_rank4.pt: 46%|████▌ | 1.13GB / 2.49GB  ...nts/optim_055000_rank5.pt: 46%|████▌ | 1.13GB / 2.49GB  ...nts/optim_055000_rank7.pt: 46%|████▌ | 1.13GB / 2.49GB  ...nts/optim_055000_rank0.pt: 45%|████▌ | 1.12GB / 2.49GB  Processing Files (0 / 8) : 37%|███▋ | 8.18GB / 22.2GB, 759MB/s New Data Upload : 60%|██████ | 2.19GB / 3.62GB, 178MB/s  ...eckpoints/model_055000.pt: 11%|█▏ | 267MB / 2.35GB  ...nts/optim_055000_rank1.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_055000_rank2.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_055000_rank3.pt: 46%|████▌ | 1.15GB / 2.49GB  ...nts/optim_055000_rank4.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_055000_rank5.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_055000_rank7.pt: 46%|████▋ | 1.15GB / 2.49GB  ...nts/optim_055000_rank0.pt: 46%|████▌ | 1.15GB / 2.49GB  Processing Files (0 / 8) : 38%|███▊ | 8.36GB / 22.2GB, 774MB/s New Data Upload : 61%|██████▏ | 2.23GB / 3.62GB, 180MB/s  ...eckpoints/model_055000.pt: 12%|█▏ | 276MB / 2.35GB  ...nts/optim_055000_rank1.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_055000_rank2.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_055000_rank3.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_055000_rank4.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_055000_rank5.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_055000_rank7.pt: 48%|████▊ | 1.18GB / 2.49GB  ...nts/optim_055000_rank0.pt: 47%|████▋ | 1.17GB / 2.49GB  Processing Files (0 / 8) : 39%|███▊ | 8.57GB / 22.2GB, 792MB/s New Data Upload : 62%|██████▏ | 2.28GB / 3.69GB, 182MB/s  ...eckpoints/model_055000.pt: 12%|█▏ | 287MB / 2.35GB  ...nts/optim_055000_rank1.pt: 49%|████▊ | 1.21GB / 2.49GB  ...nts/optim_055000_rank2.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_055000_rank3.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_055000_rank4.pt: 49%|████▉ | 1.21GB / 2.49GB  ...nts/optim_055000_rank5.pt: 49%|████▉ | 1.21GB / 2.49GB  ...nts/optim_055000_rank7.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_055000_rank0.pt: 48%|████▊ | 1.19GB / 2.49GB  Processing Files (0 / 8) : 39%|███▉ | 8.73GB / 22.2GB, 804MB/s New Data Upload : 63%|██████▎ | 2.32GB / 3.69GB, 184MB/s  ...eckpoints/model_055000.pt: 12%|█▏ | 293MB / 2.35GB  ...nts/optim_055000_rank1.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_055000_rank2.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_055000_rank3.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_055000_rank4.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_055000_rank5.pt: 50%|█████ | 1.24GB / 2.49GB  ...nts/optim_055000_rank7.pt: 50%|████▉ | 1.23GB / 2.49GB  ...nts/optim_055000_rank0.pt: 49%|████▉ | 1.22GB / 2.49GB  Processing Files (0 / 8) : 40%|████ | 8.95GB / 22.2GB, 821MB/s New Data Upload : 62%|██████▏ | 2.37GB / 3.82GB, 186MB/s  ...eckpoints/model_055000.pt: 13%|█▎ | 298MB / 2.35GB  ...nts/optim_055000_rank1.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_055000_rank2.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_055000_rank3.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_055000_rank4.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_055000_rank5.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_055000_rank7.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_055000_rank0.pt: 50%|█████ | 1.24GB / 2.49GB  Processing Files (0 / 8) : 41%|████ | 9.13GB / 22.2GB, 833MB/s New Data Upload : 63%|██████▎ | 2.41GB / 3.82GB, 187MB/s  ...eckpoints/model_055000.pt: 13%|█▎ | 305MB / 2.35GB  ...nts/optim_055000_rank1.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_055000_rank2.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_055000_rank3.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_055000_rank4.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_055000_rank5.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_055000_rank7.pt: 51%|█████▏ | 1.27GB / 2.49GB  ...nts/optim_055000_rank0.pt: 51%|█████ | 1.26GB / 2.49GB  Processing Files (0 / 8) : 42%|████▏ | 9.29GB / 22.2GB, 839MB/s New Data Upload : 64%|██████▍ | 2.44GB / 3.82GB, 188MB/s  ...eckpoints/model_055000.pt: 13%|█▎ | 313MB / 2.35GB  ...nts/optim_055000_rank1.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_055000_rank2.pt: 53%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_055000_rank3.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_055000_rank4.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_055000_rank5.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_055000_rank7.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_055000_rank0.pt: 52%|█████▏ | 1.29GB / 2.49GB  Processing Files (0 / 8) : 43%|████▎ | 9.48GB / 22.2GB, 848MB/s New Data Upload : 65%|██████▌ | 2.49GB / 3.82GB, 190MB/s  ...eckpoints/model_055000.pt: 13%|█▎ | 317MB / 2.35GB  ...nts/optim_055000_rank1.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_055000_rank2.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_055000_rank3.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_055000_rank4.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_055000_rank5.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_055000_rank7.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_055000_rank0.pt: 53%|█████▎ | 1.31GB / 2.49GB  Processing Files (0 / 8) : 43%|████▎ | 9.64GB / 22.2GB, 858MB/s New Data Upload : 64%|██████▎ | 2.52GB / 3.96GB, 192MB/s  ...eckpoints/model_055000.pt: 14%|█▎ | 322MB / 2.35GB  ...nts/optim_055000_rank1.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_055000_rank2.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_055000_rank3.pt: 54%|█████▎ | 1.34GB / 2.49GB  ...nts/optim_055000_rank4.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_055000_rank5.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_055000_rank7.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_055000_rank0.pt: 54%|█████▍ | 1.34GB / 2.49GB  Processing Files (0 / 8) : 44%|████▍ | 9.83GB / 22.2GB, 860MB/s New Data Upload : 63%|██████▎ | 2.56GB / 4.09GB, 193MB/s  ...eckpoints/model_055000.pt: 14%|█▍ | 336MB / 2.35GB  ...nts/optim_055000_rank1.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_055000_rank2.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_055000_rank3.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_055000_rank4.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_055000_rank5.pt: 56%|█████▌ | 1.40GB / 2.49GB  ...nts/optim_055000_rank7.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_055000_rank0.pt: 55%|█████▍ | 1.36GB / 2.49GB  Processing Files (0 / 8) : 45%|████▌ | 10.0GB / 22.2GB, 861MB/s New Data Upload : 62%|██████▏ | 2.61GB / 4.22GB, 194MB/s  ...eckpoints/model_055000.pt: 15%|█▍ | 342MB / 2.35GB  ...nts/optim_055000_rank1.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_055000_rank2.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_055000_rank3.pt: 56%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_055000_rank4.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_055000_rank5.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_055000_rank7.pt: 56%|█████▌ | 1.40GB / 2.49GB  ...nts/optim_055000_rank0.pt: 56%|█████▌ | 1.39GB / 2.49GB  Processing Files (0 / 8) : 46%|████▌ | 10.2GB / 22.2GB, 862MB/s New Data Upload : 62%|██████▏ | 2.65GB / 4.29GB, 195MB/s  ...eckpoints/model_055000.pt: 15%|█▍ | 346MB / 2.35GB  ...nts/optim_055000_rank1.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_055000_rank2.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_055000_rank3.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...nts/optim_055000_rank4.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_055000_rank5.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_055000_rank7.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_055000_rank0.pt: 57%|█████▋ | 1.41GB / 2.49GB  Processing Files (0 / 8) : 47%|████▋ | 10.4GB / 22.2GB, 868MB/s New Data Upload : 63%|██████▎ | 2.68GB / 4.29GB, 196MB/s  ...eckpoints/model_055000.pt: 15%|█▍ | 352MB / 2.35GB  ...nts/optim_055000_rank1.pt: 59%|█████▉ | 1.46GB / 2.49GB  ...nts/optim_055000_rank2.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_055000_rank3.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_055000_rank4.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_055000_rank5.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_055000_rank7.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_055000_rank0.pt: 58%|█████▊ | 1.43GB / 2.49GB  Processing Files (0 / 8) : 47%|████▋ | 10.5GB / 22.2GB, 863MB/s New Data Upload : 61%|██████▏ | 2.72GB / 4.43GB, 196MB/s  ...eckpoints/model_055000.pt: 15%|█▌ | 363MB / 2.35GB  ...nts/optim_055000_rank1.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_055000_rank2.pt: 61%|██████ | 1.50GB / 2.49GB  ...nts/optim_055000_rank3.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_055000_rank4.pt: 60%|██████ | 1.49GB / 2.49GB  ...nts/optim_055000_rank5.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_055000_rank7.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_055000_rank0.pt: 59%|█████▉ | 1.46GB / 2.49GB  Processing Files (0 / 8) : 48%|████▊ | 10.7GB / 22.2GB, 869MB/s New Data Upload : 63%|██████▎ | 2.77GB / 4.43GB, 198MB/s  ...eckpoints/model_055000.pt: 16%|█▌ | 375MB / 2.35GB  ...nts/optim_055000_rank1.pt: 61%|██████ | 1.52GB / 2.49GB  ...nts/optim_055000_rank2.pt: 61%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_055000_rank3.pt: 59%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_055000_rank4.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_055000_rank5.pt: 61%|██████ | 1.52GB / 2.49GB  ...nts/optim_055000_rank7.pt: 60%|██████ | 1.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 60%|█████▉ | 1.48GB / 2.49GB  Processing Files (0 / 8) : 49%|████▉ | 10.9GB / 22.2GB, 869MB/s New Data Upload : 63%|██████▎ | 2.82GB / 4.49GB, 199MB/s  ...eckpoints/model_055000.pt: 16%|█▋ | 384MB / 2.35GB  ...nts/optim_055000_rank1.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_055000_rank2.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_055000_rank3.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_055000_rank4.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_055000_rank5.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_055000_rank7.pt: 61%|██████ | 1.52GB / 2.49GB  ...nts/optim_055000_rank0.pt: 61%|██████ | 1.51GB / 2.49GB  Processing Files (0 / 8) : 50%|████▉ | 11.1GB / 22.2GB, 873MB/s New Data Upload : 64%|██████▎ | 2.86GB / 4.49GB, 200MB/s  ...eckpoints/model_055000.pt: 17%|█▋ | 396MB / 2.35GB  ...nts/optim_055000_rank1.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_055000_rank2.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_055000_rank3.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_055000_rank4.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_055000_rank5.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_055000_rank7.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_055000_rank0.pt: 62%|██████▏ | 1.53GB / 2.49GB  Processing Files (0 / 8) : 51%|█████ | 11.3GB / 22.2GB, 874MB/s New Data Upload : 64%|██████▍ | 2.91GB / 4.56GB, 202MB/s  ...nts/optim_055000_rank6.pt: 3%|▎ | 79.9MB / 2.49GB  ...eckpoints/model_055000.pt: 17%|█▋ | 404MB / 2.35GB  ...nts/optim_055000_rank1.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_055000_rank2.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_055000_rank3.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_055000_rank4.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_055000_rank5.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_055000_rank7.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_055000_rank0.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_055000_rank6.pt: 3%|▎ | 79.9MB / 2.49GB  Processing Files (0 / 9) : 52%|█████▏ | 11.5GB / 22.2GB, 879MB/s New Data Upload : 63%|██████▎ | 2.95GB / 4.68GB, 202MB/s  ...eckpoints/model_055000.pt: 18%|█▊ | 414MB / 2.35GB  ...nts/optim_055000_rank1.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_055000_rank2.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_055000_rank3.pt: 64%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_055000_rank4.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_055000_rank5.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_055000_rank7.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_055000_rank0.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_055000_rank6.pt: 9%|▉ | 224MB / 2.49GB  Processing Files (0 / 9) : 53%|█████▎ | 11.9GB / 22.2GB, 893MB/s New Data Upload : 64%|██████▍ | 3.00GB / 4.68GB, 203MB/s  ...eckpoints/model_055000.pt: 18%|█▊ | 421MB / 2.35GB  ...nts/optim_055000_rank1.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_055000_rank2.pt: 66%|██████▌ | 1.65GB / 2.49GB  ...nts/optim_055000_rank3.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_055000_rank4.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_055000_rank5.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_055000_rank7.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_055000_rank0.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_055000_rank6.pt: 14%|█▍ | 359MB / 2.49GB  Processing Files (0 / 9) : 55%|█████▍ | 12.2GB / 22.2GB, 903MB/s New Data Upload : 65%|██████▌ | 3.04GB / 4.68GB, 203MB/s  ...eckpoints/model_055000.pt: 18%|█▊ | 430MB / 2.35GB  ...nts/optim_055000_rank1.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_055000_rank2.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_055000_rank3.pt: 65%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_055000_rank4.pt: 67%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_055000_rank5.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_055000_rank7.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_055000_rank0.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_055000_rank6.pt: 20%|█▉ | 487MB / 2.49GB  Processing Files (0 / 9) : 56%|█████▌ | 12.5GB / 22.2GB, 909MB/s New Data Upload : 66%|██████▌ | 3.09GB / 4.68GB, 203MB/s  ...eckpoints/model_055000.pt: 19%|█▉ | 443MB / 2.35GB  ...nts/optim_055000_rank1.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_055000_rank2.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_055000_rank3.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_055000_rank4.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_055000_rank5.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_055000_rank7.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_055000_rank0.pt: 67%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_055000_rank6.pt: 25%|██▍ | 610MB / 2.49GB  Processing Files (0 / 9) : 57%|█████▋ | 12.7GB / 22.2GB, 919MB/s New Data Upload : 67%|██████▋ | 3.13GB / 4.68GB, 203MB/s  ...eckpoints/model_055000.pt: 19%|█▉ | 448MB / 2.35GB  ...nts/optim_055000_rank1.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_055000_rank2.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_055000_rank3.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_055000_rank4.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_055000_rank5.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_055000_rank7.pt: 67%|██████▋ | 1.68GB / 2.49GB  ...nts/optim_055000_rank0.pt: 67%|██████▋ | 1.68GB / 2.49GB  ...nts/optim_055000_rank6.pt: 30%|███ | 751MB / 2.49GB  Processing Files (0 / 9) : 59%|█████▊ | 13.0GB / 22.2GB, 931MB/s New Data Upload : 68%|██████▊ | 3.16GB / 4.68GB, 203MB/s  ...eckpoints/model_055000.pt: 19%|█▉ | 454MB / 2.35GB  ...nts/optim_055000_rank1.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_055000_rank2.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_055000_rank3.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_055000_rank4.pt: 69%|██████▊ | 1.71GB / 2.49GB  ...nts/optim_055000_rank5.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_055000_rank7.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_055000_rank0.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_055000_rank6.pt: 34%|███▎ | 833MB / 2.49GB  Processing Files (0 / 9) : 60%|█████▉ | 13.2GB / 22.2GB, 940MB/s New Data Upload : 68%|██████▊ | 3.20GB / 4.68GB, 203MB/s  ...eckpoints/model_055000.pt: 20%|█▉ | 460MB / 2.35GB  ...nts/optim_055000_rank1.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_055000_rank2.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_055000_rank3.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_055000_rank4.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_055000_rank5.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_055000_rank7.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_055000_rank0.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_055000_rank6.pt: 37%|███▋ | 916MB / 2.49GB  Processing Files (0 / 9) : 60%|██████ | 13.4GB / 22.2GB, 948MB/s New Data Upload : 69%|██████▉ | 3.23GB / 4.68GB, 203MB/s  ...eckpoints/model_055000.pt: 20%|█▉ | 464MB / 2.35GB  ...nts/optim_055000_rank1.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_055000_rank2.pt: 71%|███████▏ | 1.77GB / 2.49GB  ...nts/optim_055000_rank3.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_055000_rank4.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_055000_rank5.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_055000_rank7.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_055000_rank0.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_055000_rank6.pt: 45%|████▍ | 1.11GB / 2.49GB  Processing Files (0 / 9) : 62%|██████▏ | 13.8GB / 22.2GB, 971MB/s New Data Upload : 70%|███████ | 3.27GB / 4.68GB, 204MB/s  ...eckpoints/model_055000.pt: 20%|█▉ | 470MB / 2.35GB  ...nts/optim_055000_rank1.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_055000_rank2.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_055000_rank3.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_055000_rank4.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_055000_rank5.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_055000_rank7.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_055000_rank0.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_055000_rank6.pt: 46%|████▌ | 1.14GB / 2.49GB  Processing Files (0 / 9) : 63%|██████▎ | 14.0GB / 22.2GB, 973MB/s New Data Upload : 71%|███████ | 3.31GB / 4.68GB, 203MB/s  ...eckpoints/model_055000.pt: 20%|██ | 478MB / 2.35GB  ...nts/optim_055000_rank1.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_055000_rank2.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_055000_rank3.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_055000_rank4.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_055000_rank5.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_055000_rank7.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_055000_rank0.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_055000_rank6.pt: 51%|█████ | 1.27GB / 2.49GB  Processing Files (0 / 9) : 64%|██████▍ | 14.3GB / 22.2GB, 982MB/s New Data Upload : 71%|███████ | 3.35GB / 4.74GB, 203MB/s  ...eckpoints/model_055000.pt: 21%|██ | 483MB / 2.35GB  ...nts/optim_055000_rank1.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_055000_rank2.pt: 73%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_055000_rank3.pt: 71%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_055000_rank4.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_055000_rank5.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_055000_rank7.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_055000_rank0.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_055000_rank6.pt: 56%|█████▌ | 1.38GB / 2.49GB  Processing Files (0 / 9) : 65%|██████▌ | 14.5GB / 22.2GB, 987MB/s New Data Upload : 70%|███████ | 3.38GB / 4.81GB, 202MB/s  ...eckpoints/model_055000.pt: 21%|██ | 490MB / 2.35GB  ...nts/optim_055000_rank1.pt: 74%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_055000_rank2.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_055000_rank3.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_055000_rank4.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_055000_rank5.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_055000_rank7.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_055000_rank0.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_055000_rank6.pt: 57%|█████▋ | 1.42GB / 2.49GB  Processing Files (0 / 9) : 66%|██████▌ | 14.6GB / 22.2GB, 981MB/s New Data Upload : 71%|███████ | 3.41GB / 4.81GB, 200MB/s  ...eckpoints/model_055000.pt: 21%|██ | 494MB / 2.35GB  ...nts/optim_055000_rank1.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_055000_rank2.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_055000_rank3.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_055000_rank4.pt: 74%|███████▍ | 1.83GB / 2.49GB  ...nts/optim_055000_rank5.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_055000_rank7.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_055000_rank0.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_055000_rank6.pt: 62%|██████▏ | 1.55GB / 2.49GB  Processing Files (0 / 9) : 67%|██████▋ | 14.9GB / 22.2GB, 995MB/s New Data Upload : 71%|███████ | 3.45GB / 4.88GB, 200MB/s  ...eckpoints/model_055000.pt: 21%|██▏ | 505MB / 2.35GB  ...nts/optim_055000_rank1.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_055000_rank2.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_055000_rank3.pt: 74%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_055000_rank4.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_055000_rank5.pt: 75%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_055000_rank7.pt: 74%|███████▍ | 1.83GB / 2.49GB  ...nts/optim_055000_rank0.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_055000_rank6.pt: 64%|██████▍ | 1.59GB / 2.49GB  Processing Files (0 / 9) : 68%|██████▊ | 15.1GB / 22.2GB, 997MB/s New Data Upload : 71%|███████▏ | 3.48GB / 4.88GB, 200MB/s  ...eckpoints/model_055000.pt: 22%|██▏ | 511MB / 2.35GB  ...nts/optim_055000_rank1.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_055000_rank2.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_055000_rank3.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_055000_rank4.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_055000_rank5.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_055000_rank7.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_055000_rank0.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_055000_rank6.pt: 68%|██████▊ | 1.70GB / 2.49GB  Processing Files (0 / 9) : 69%|██████▉ | 15.3GB / 22.2GB, 1.01GB/s New Data Upload : 71%|███████ | 3.51GB / 4.94GB, 200MB/s  ...eckpoints/model_055000.pt: 22%|██▏ | 527MB / 2.35GB  ...nts/optim_055000_rank1.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_055000_rank2.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_055000_rank3.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_055000_rank4.pt: 75%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_055000_rank5.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_055000_rank7.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_055000_rank0.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_055000_rank6.pt: 71%|███████ | 1.77GB / 2.49GB  Processing Files (0 / 9) : 70%|██████▉ | 15.5GB / 22.2GB, 1.00GB/s New Data Upload : 72%|███████▏ | 3.55GB / 4.94GB, 199MB/s  ...eckpoints/model_055000.pt: 23%|██▎ | 537MB / 2.35GB  ...nts/optim_055000_rank1.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_055000_rank2.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_055000_rank3.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_055000_rank4.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_055000_rank5.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_055000_rank7.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_055000_rank0.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_055000_rank6.pt: 75%|███████▍ | 1.85GB / 2.49GB  Processing Files (0 / 9) : 71%|███████ | 15.7GB / 22.2GB, 1.01GB/s New Data Upload : 72%|███████▏ | 3.58GB / 5.01GB, 199MB/s  ...eckpoints/model_055000.pt: 23%|██▎ | 547MB / 2.35GB  ...nts/optim_055000_rank1.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_055000_rank2.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_055000_rank3.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_055000_rank4.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_055000_rank5.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_055000_rank7.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_055000_rank0.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_055000_rank6.pt: 76%|███████▋ | 1.90GB / 2.49GB  Processing Files (0 / 9) : 71%|███████▏ | 15.8GB / 22.2GB, 1.01GB/s New Data Upload : 72%|███████▏ | 3.62GB / 5.01GB, 200MB/s  ...eckpoints/model_055000.pt: 24%|██▍ | 569MB / 2.35GB  ...nts/optim_055000_rank1.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_055000_rank2.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_055000_rank3.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_055000_rank4.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_055000_rank5.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_055000_rank7.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_055000_rank0.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_055000_rank6.pt: 77%|███████▋ | 1.92GB / 2.49GB  Processing Files (0 / 9) : 72%|███████▏ | 16.0GB / 22.2GB, 1.01GB/s New Data Upload : 72%|███████▏ | 3.66GB / 5.08GB, 200MB/s  ...eckpoints/model_055000.pt: 25%|██▍ | 587MB / 2.35GB  ...nts/optim_055000_rank1.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_055000_rank2.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_055000_rank3.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_055000_rank4.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_055000_rank5.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_055000_rank7.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_055000_rank0.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_055000_rank6.pt: 78%|███████▊ | 1.94GB / 2.49GB  Processing Files (0 / 9) : 72%|███████▏ | 16.1GB / 22.2GB, 998MB/s New Data Upload : 73%|███████▎ | 3.70GB / 5.08GB, 199MB/s  ...eckpoints/model_055000.pt: 26%|██▌ | 601MB / 2.35GB  ...nts/optim_055000_rank1.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_055000_rank2.pt: 80%|███████▉ | 1.99GB / 2.49GB  ...nts/optim_055000_rank3.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_055000_rank4.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_055000_rank5.pt: 79%|███████▊ | 1.96GB / 2.49GB  ...nts/optim_055000_rank7.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_055000_rank0.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_055000_rank6.pt: 78%|███████▊ | 1.95GB / 2.49GB  Processing Files (0 / 9) : 73%|███████▎ | 16.2GB / 22.2GB, 991MB/s New Data Upload : 73%|███████▎ | 3.73GB / 5.15GB, 198MB/s  ...eckpoints/model_055000.pt: 26%|██▋ | 620MB / 2.35GB  ...nts/optim_055000_rank1.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_055000_rank2.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_055000_rank3.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_055000_rank4.pt: 79%|███████▊ | 1.96GB / 2.49GB  ...nts/optim_055000_rank5.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_055000_rank7.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_055000_rank0.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_055000_rank6.pt: 79%|███████▉ | 1.96GB / 2.49GB  Processing Files (0 / 9) : 74%|███████▎ | 16.3GB / 22.2GB, 987MB/s New Data Upload : 72%|███████▏ | 3.77GB / 5.21GB, 199MB/s  ...eckpoints/model_055000.pt: 27%|██▋ | 637MB / 2.35GB  ...nts/optim_055000_rank1.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_055000_rank2.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_055000_rank3.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_055000_rank4.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_055000_rank5.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_055000_rank7.pt: 79%|███████▊ | 1.96GB / 2.49GB  ...nts/optim_055000_rank0.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_055000_rank6.pt: 80%|███████▉ | 1.98GB / 2.49GB  Processing Files (0 / 9) : 74%|███████▍ | 16.5GB / 22.2GB, 983MB/s New Data Upload : 72%|███████▏ | 3.81GB / 5.28GB, 199MB/s  ...eckpoints/model_055000.pt: 28%|██▊ | 655MB / 2.35GB  ...nts/optim_055000_rank1.pt: 81%|████████ | 2.00GB / 2.49GB  ...nts/optim_055000_rank2.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_055000_rank3.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_055000_rank4.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_055000_rank5.pt: 80%|███████▉ | 1.99GB / 2.49GB  ...nts/optim_055000_rank7.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_055000_rank0.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_055000_rank6.pt: 80%|███████▉ | 1.99GB / 2.49GB  Processing Files (0 / 9) : 75%|███████▍ | 16.6GB / 22.2GB, 976MB/s New Data Upload : 73%|███████▎ | 3.84GB / 5.28GB, 198MB/s  ...eckpoints/model_055000.pt: 29%|██▊ | 671MB / 2.35GB  ...nts/optim_055000_rank1.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_055000_rank2.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_055000_rank3.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_055000_rank4.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_055000_rank5.pt: 81%|████████ | 2.00GB / 2.49GB  ...nts/optim_055000_rank7.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_055000_rank0.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_055000_rank6.pt: 81%|████████ | 2.00GB / 2.49GB  Processing Files (0 / 9) : 75%|███████▌ | 16.7GB / 22.2GB, 972MB/s New Data Upload : 73%|███████▎ | 3.88GB / 5.35GB, 198MB/s  ...eckpoints/model_055000.pt: 30%|██▉ | 694MB / 2.35GB  ...nts/optim_055000_rank1.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_055000_rank2.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_055000_rank3.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_055000_rank4.pt: 81%|████████ | 2.00GB / 2.49GB  ...nts/optim_055000_rank5.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_055000_rank7.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_055000_rank0.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_055000_rank6.pt: 81%|████████ | 2.01GB / 2.49GB  Processing Files (0 / 9) : 76%|███████▌ | 16.8GB / 22.2GB, 968MB/s New Data Upload : 72%|███████▏ | 3.92GB / 5.41GB, 199MB/s  ...eckpoints/model_055000.pt: 31%|███ | 718MB / 2.35GB  ...nts/optim_055000_rank1.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_055000_rank2.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_055000_rank3.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_055000_rank4.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_055000_rank5.pt: 81%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_055000_rank7.pt: 81%|████████ | 2.00GB / 2.49GB  ...nts/optim_055000_rank0.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_055000_rank6.pt: 82%|████████▏ | 2.03GB / 2.49GB  Processing Files (0 / 9) : 76%|███████▌ | 16.9GB / 22.2GB, 966MB/s New Data Upload : 72%|███████▏ | 3.96GB / 5.48GB, 199MB/s  ...eckpoints/model_055000.pt: 31%|███▏ | 738MB / 2.35GB  ...nts/optim_055000_rank1.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_055000_rank2.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_055000_rank3.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_055000_rank4.pt: 81%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_055000_rank5.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_055000_rank7.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_055000_rank0.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_055000_rank6.pt: 82%|████████▏ | 2.03GB / 2.49GB  Processing Files (0 / 9) : 77%|███████▋ | 17.0GB / 22.2GB, 954MB/s New Data Upload : 72%|███████▏ | 3.99GB / 5.55GB, 198MB/s  ...eckpoints/model_055000.pt: 33%|███▎ | 768MB / 2.35GB  ...nts/optim_055000_rank1.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_055000_rank2.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_055000_rank3.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_055000_rank4.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_055000_rank5.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_055000_rank7.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_055000_rank0.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_055000_rank6.pt: 82%|████████▏ | 2.05GB / 2.49GB  Processing Files (0 / 9) : 77%|███████▋ | 17.1GB / 22.2GB, 948MB/s New Data Upload : 73%|███████▎ | 4.04GB / 5.55GB, 199MB/s  ...eckpoints/model_055000.pt: 33%|███▎ | 785MB / 2.35GB  ...nts/optim_055000_rank1.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_055000_rank2.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_055000_rank3.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_055000_rank4.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_055000_rank5.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_055000_rank7.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_055000_rank0.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_055000_rank6.pt: 83%|████████▎ | 2.06GB / 2.49GB  Processing Files (0 / 9) : 78%|███████▊ | 17.2GB / 22.2GB, 944MB/s New Data Upload : 73%|███████▎ | 4.07GB / 5.55GB, 198MB/s  ...eckpoints/model_055000.pt: 35%|███▍ | 812MB / 2.35GB  ...nts/optim_055000_rank1.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_055000_rank2.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_055000_rank3.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_055000_rank4.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_055000_rank5.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_055000_rank7.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_055000_rank0.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_055000_rank6.pt: 83%|████████▎ | 2.07GB / 2.49GB  Processing Files (0 / 9) : 78%|███████▊ | 17.4GB / 22.2GB, 938MB/s New Data Upload : 74%|███████▍ | 4.12GB / 5.55GB, 198MB/s  ...eckpoints/model_055000.pt: 36%|███▌ | 838MB / 2.35GB  ...nts/optim_055000_rank1.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_055000_rank2.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_055000_rank3.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_055000_rank4.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_055000_rank5.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_055000_rank7.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_055000_rank0.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_055000_rank6.pt: 84%|████████▎ | 2.08GB / 2.49GB  Processing Files (0 / 9) : 78%|███████▊ | 17.5GB / 22.2GB, 927MB/s New Data Upload : 74%|███████▍ | 4.16GB / 5.62GB, 197MB/s  ...eckpoints/model_055000.pt: 37%|███▋ | 867MB / 2.35GB  ...nts/optim_055000_rank1.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_055000_rank2.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_055000_rank3.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_055000_rank4.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_055000_rank5.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_055000_rank7.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_055000_rank0.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_055000_rank6.pt: 84%|████████▍ | 2.09GB / 2.49GB  Processing Files (0 / 9) : 79%|███████▉ | 17.6GB / 22.2GB, 924MB/s New Data Upload : 75%|███████▍ | 4.21GB / 5.62GB, 198MB/s  ...eckpoints/model_055000.pt: 38%|███▊ | 886MB / 2.35GB  ...nts/optim_055000_rank1.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_055000_rank2.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_055000_rank3.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_055000_rank4.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_055000_rank5.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_055000_rank7.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_055000_rank0.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_055000_rank6.pt: 84%|████████▍ | 2.10GB / 2.49GB  Processing Files (0 / 9) : 79%|███████▉ | 17.7GB / 22.2GB, 913MB/s New Data Upload : 75%|███████▍ | 4.23GB / 5.68GB, 197MB/s  ...eckpoints/model_055000.pt: 39%|███▉ | 924MB / 2.35GB  ...nts/optim_055000_rank1.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_055000_rank2.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_055000_rank3.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_055000_rank4.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_055000_rank5.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_055000_rank7.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_055000_rank0.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_055000_rank6.pt: 85%|████████▍ | 2.11GB / 2.49GB  Processing Files (0 / 9) : 80%|████████ | 17.8GB / 22.2GB, 904MB/s New Data Upload : 75%|███████▍ | 4.29GB / 5.75GB, 197MB/s  ...eckpoints/model_055000.pt: 41%|████ | 955MB / 2.35GB  ...nts/optim_055000_rank1.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_055000_rank2.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_055000_rank3.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_055000_rank4.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_055000_rank5.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_055000_rank7.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_055000_rank0.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_055000_rank6.pt: 85%|████████▌ | 2.12GB / 2.49GB  Processing Files (0 / 9) : 81%|████████ | 17.9GB / 22.2GB, 901MB/s New Data Upload : 75%|███████▌ | 4.34GB / 5.75GB, 198MB/s  ...eckpoints/model_055000.pt: 42%|████▏ | 979MB / 2.35GB  ...nts/optim_055000_rank1.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_055000_rank2.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_055000_rank3.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_055000_rank4.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_055000_rank5.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_055000_rank7.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_055000_rank0.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_055000_rank6.pt: 86%|████████▌ | 2.13GB / 2.49GB  Processing Files (0 / 9) : 81%|████████ | 18.0GB / 22.2GB, 889MB/s New Data Upload : 75%|███████▌ | 4.38GB / 5.82GB, 197MB/s  ...eckpoints/model_055000.pt: 43%|████▎ | 1.01GB / 2.35GB  ...nts/optim_055000_rank1.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_055000_rank2.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_055000_rank3.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_055000_rank4.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_055000_rank5.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_055000_rank7.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_055000_rank0.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_055000_rank6.pt: 86%|████████▌ | 2.14GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.1GB / 22.2GB, 882MB/s New Data Upload : 75%|███████▌ | 4.42GB / 5.88GB, 197MB/s  ...eckpoints/model_055000.pt: 44%|████▍ | 1.04GB / 2.35GB  ...nts/optim_055000_rank1.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_055000_rank2.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_055000_rank3.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_055000_rank4.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_055000_rank5.pt: 86%|████████▋ | 2.14GB / 2.49GB  ...nts/optim_055000_rank7.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_055000_rank0.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_055000_rank6.pt: 87%|████████▋ | 2.16GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.3GB / 22.2GB, 879MB/s New Data Upload : 76%|███████▌ | 4.47GB / 5.88GB, 199MB/s  ...eckpoints/model_055000.pt: 45%|████▌ | 1.07GB / 2.35GB  ...nts/optim_055000_rank1.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_055000_rank2.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_055000_rank3.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_055000_rank4.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_055000_rank5.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_055000_rank7.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_055000_rank0.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_055000_rank6.pt: 87%|████████▋ | 2.16GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.3GB / 22.2GB, 867MB/s New Data Upload : 76%|███████▌ | 4.50GB / 5.95GB, 198MB/s  ...eckpoints/model_055000.pt: 46%|████▋ | 1.09GB / 2.35GB  ...nts/optim_055000_rank1.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_055000_rank2.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_055000_rank3.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_055000_rank4.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_055000_rank5.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_055000_rank7.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_055000_rank0.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_055000_rank6.pt: 88%|████████▊ | 2.18GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.5GB / 22.2GB, 866MB/s New Data Upload : 76%|███████▌ | 4.54GB / 6.02GB, 198MB/s  ...eckpoints/model_055000.pt: 48%|████▊ | 1.12GB / 2.35GB  ...nts/optim_055000_rank1.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_055000_rank2.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_055000_rank3.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_055000_rank4.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_055000_rank5.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_055000_rank7.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_055000_rank0.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_055000_rank6.pt: 88%|████████▊ | 2.18GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.6GB / 22.2GB, 856MB/s New Data Upload : 75%|███████▌ | 4.59GB / 6.09GB, 199MB/s  ...eckpoints/model_055000.pt: 49%|████▉ | 1.15GB / 2.35GB  ...nts/optim_055000_rank1.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_055000_rank2.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_055000_rank3.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_055000_rank4.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_055000_rank5.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_055000_rank7.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_055000_rank0.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_055000_rank6.pt: 88%|████████▊ | 2.19GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▍ | 18.7GB / 22.2GB, 849MB/s New Data Upload : 76%|███████▌ | 4.63GB / 6.09GB, 198MB/s  ...eckpoints/model_055000.pt: 50%|█████ | 1.18GB / 2.35GB  ...nts/optim_055000_rank1.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_055000_rank2.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_055000_rank3.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_055000_rank4.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_055000_rank5.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_055000_rank7.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_055000_rank0.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_055000_rank6.pt: 89%|████████▊ | 2.21GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▍ | 18.8GB / 22.2GB, 843MB/s New Data Upload : 76%|███████▌ | 4.68GB / 6.15GB, 199MB/s  ...eckpoints/model_055000.pt: 52%|█████▏ | 1.22GB / 2.35GB  ...nts/optim_055000_rank1.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_055000_rank2.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_055000_rank3.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_055000_rank4.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_055000_rank5.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_055000_rank7.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_055000_rank0.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_055000_rank6.pt: 89%|████████▉ | 2.21GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▍ | 18.9GB / 22.2GB, 834MB/s New Data Upload : 76%|███████▌ | 4.73GB / 6.22GB, 201MB/s  ...eckpoints/model_055000.pt: 53%|█████▎ | 1.25GB / 2.35GB  ...nts/optim_055000_rank1.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_055000_rank2.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_055000_rank3.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_055000_rank4.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_055000_rank5.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_055000_rank7.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_055000_rank0.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_055000_rank6.pt: 89%|████████▉ | 2.22GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▌ | 19.0GB / 22.2GB, 831MB/s New Data Upload : 77%|███████▋ | 4.78GB / 6.22GB, 202MB/s  ...eckpoints/model_055000.pt: 55%|█████▍ | 1.28GB / 2.35GB  ...nts/optim_055000_rank1.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_055000_rank2.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_055000_rank3.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_055000_rank4.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_055000_rank5.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_055000_rank7.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_055000_rank0.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_055000_rank6.pt: 90%|████████▉ | 2.23GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▌ | 19.1GB / 22.2GB, 817MB/s New Data Upload : 78%|███████▊ | 4.82GB / 6.22GB, 201MB/s  ...eckpoints/model_055000.pt: 56%|█████▌ | 1.31GB / 2.35GB  ...nts/optim_055000_rank1.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_055000_rank2.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_055000_rank3.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_055000_rank4.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_055000_rank5.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_055000_rank7.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_055000_rank0.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_055000_rank6.pt: 90%|█████████ | 2.24GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▋ | 19.2GB / 22.2GB, 812MB/s New Data Upload : 78%|███████▊ | 4.87GB / 6.22GB, 201MB/s  ...eckpoints/model_055000.pt: 58%|█████▊ | 1.36GB / 2.35GB  ...nts/optim_055000_rank1.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_055000_rank2.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_055000_rank3.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_055000_rank4.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_055000_rank5.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_055000_rank7.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_055000_rank0.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_055000_rank6.pt: 91%|█████████ | 2.25GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.3GB / 22.2GB, 807MB/s New Data Upload : 78%|███████▊ | 4.92GB / 6.33GB, 202MB/s  ...eckpoints/model_055000.pt: 59%|█████▉ | 1.39GB / 2.35GB  ...nts/optim_055000_rank1.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_055000_rank2.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_055000_rank3.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_055000_rank4.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_055000_rank5.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_055000_rank7.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_055000_rank0.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_055000_rank6.pt: 91%|█████████ | 2.26GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.4GB / 22.2GB, 796MB/s New Data Upload : 79%|███████▊ | 4.97GB / 6.33GB, 202MB/s  ...eckpoints/model_055000.pt: 60%|██████ | 1.42GB / 2.35GB  ...nts/optim_055000_rank1.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_055000_rank2.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_055000_rank3.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_055000_rank4.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_055000_rank5.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_055000_rank7.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_055000_rank0.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_055000_rank6.pt: 91%|█████████ | 2.27GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.5GB / 22.2GB, 783MB/s New Data Upload : 79%|███████▉ | 5.01GB / 6.33GB, 202MB/s  ...eckpoints/model_055000.pt: 62%|██████▏ | 1.45GB / 2.35GB  ...nts/optim_055000_rank1.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_055000_rank2.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_055000_rank3.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_055000_rank4.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_055000_rank5.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_055000_rank7.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_055000_rank0.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_055000_rank6.pt: 92%|█████████▏| 2.28GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.6GB / 22.2GB, 759MB/s New Data Upload : 80%|███████▉ | 5.05GB / 6.33GB, 201MB/s  ...eckpoints/model_055000.pt: 63%|██████▎ | 1.49GB / 2.35GB  ...nts/optim_055000_rank1.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_055000_rank2.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_055000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_055000_rank4.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_055000_rank5.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_055000_rank7.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_055000_rank0.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_055000_rank6.pt: 92%|█████████▏| 2.29GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▊ | 19.7GB / 22.2GB, 741MB/s New Data Upload : 81%|████████ | 5.11GB / 6.33GB, 203MB/s  ...eckpoints/model_055000.pt: 64%|██████▍ | 1.51GB / 2.35GB  ...nts/optim_055000_rank1.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_055000_rank2.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_055000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_055000_rank4.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_055000_rank5.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_055000_rank7.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_055000_rank0.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_055000_rank6.pt: 92%|█████████▏| 2.29GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▉ | 19.8GB / 22.2GB, 720MB/s New Data Upload : 81%|████████▏ | 5.15GB / 6.33GB, 202MB/s  ...eckpoints/model_055000.pt: 65%|██████▌ | 1.54GB / 2.35GB  ...nts/optim_055000_rank1.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_055000_rank2.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_055000_rank3.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_055000_rank4.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_055000_rank5.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_055000_rank7.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_055000_rank0.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_055000_rank6.pt: 93%|█████████▎| 2.30GB / 2.49GB  Processing Files (0 / 9) : 90%|████████▉ | 19.9GB / 22.2GB, 705MB/s New Data Upload : 82%|████████▏ | 5.19GB / 6.33GB, 202MB/s  ...eckpoints/model_055000.pt: 67%|██████▋ | 1.56GB / 2.35GB  ...nts/optim_055000_rank1.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_055000_rank2.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_055000_rank3.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_055000_rank4.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_055000_rank5.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_055000_rank7.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_055000_rank0.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_055000_rank6.pt: 93%|█████████▎| 2.31GB / 2.49GB  Processing Files (0 / 9) : 90%|████████▉ | 20.0GB / 22.2GB, 684MB/s New Data Upload : 83%|████████▎ | 5.23GB / 6.33GB, 203MB/s  ...eckpoints/model_055000.pt: 68%|██████▊ | 1.59GB / 2.35GB  ...nts/optim_055000_rank1.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_055000_rank2.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_055000_rank3.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_055000_rank4.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_055000_rank5.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_055000_rank7.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_055000_rank0.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_055000_rank6.pt: 93%|█████████▎| 2.32GB / 2.49GB  Processing Files (0 / 9) : 90%|█████████ | 20.1GB / 22.2GB, 670MB/s New Data Upload : 83%|████████▎ | 5.27GB / 6.33GB, 203MB/s  ...eckpoints/model_055000.pt: 69%|██████▉ | 1.62GB / 2.35GB  ...nts/optim_055000_rank1.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_055000_rank2.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_055000_rank3.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_055000_rank4.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_055000_rank5.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_055000_rank7.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_055000_rank0.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_055000_rank6.pt: 94%|█████████▎| 2.33GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.2GB / 22.2GB, 661MB/s New Data Upload : 84%|████████▍ | 5.31GB / 6.33GB, 204MB/s  ...eckpoints/model_055000.pt: 70%|██████▉ | 1.64GB / 2.35GB  ...nts/optim_055000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_055000_rank2.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_055000_rank3.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_055000_rank4.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_055000_rank5.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_055000_rank7.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_055000_rank0.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_055000_rank6.pt: 94%|█████████▍| 2.33GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.3GB / 22.2GB, 633MB/s New Data Upload : 85%|████████▍ | 5.36GB / 6.33GB, 204MB/s  ...eckpoints/model_055000.pt: 71%|███████ | 1.67GB / 2.35GB  ...nts/optim_055000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_055000_rank2.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_055000_rank3.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_055000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_055000_rank5.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_055000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_055000_rank0.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_055000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.4GB / 22.2GB, 624MB/s New Data Upload : 85%|████████▌ | 5.40GB / 6.33GB, 204MB/s  ...eckpoints/model_055000.pt: 72%|███████▏ | 1.69GB / 2.35GB  ...nts/optim_055000_rank1.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_055000_rank2.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_055000_rank3.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_055000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_055000_rank5.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_055000_rank7.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_055000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_055000_rank6.pt: 94%|█████████▍| 2.35GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.4GB / 22.2GB, 606MB/s New Data Upload : 86%|████████▌ | 5.42GB / 6.33GB, 204MB/s  ...eckpoints/model_055000.pt: 73%|███████▎ | 1.71GB / 2.35GB  ...nts/optim_055000_rank1.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_055000_rank2.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_055000_rank3.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_055000_rank4.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_055000_rank5.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_055000_rank7.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_055000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_055000_rank6.pt: 95%|█████████▍| 2.36GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.5GB / 22.2GB, 593MB/s New Data Upload : 86%|████████▋ | 5.46GB / 6.33GB, 205MB/s  ...eckpoints/model_055000.pt: 74%|███████▍ | 1.74GB / 2.35GB  ...nts/optim_055000_rank1.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_055000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_055000_rank3.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_055000_rank4.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_055000_rank5.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_055000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_055000_rank0.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_055000_rank6.pt: 95%|█████████▍| 2.36GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.6GB / 22.2GB, 582MB/s New Data Upload : 87%|████████▋ | 5.49GB / 6.33GB, 204MB/s  ...eckpoints/model_055000.pt: 75%|███████▍ | 1.76GB / 2.35GB  ...nts/optim_055000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_055000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_055000_rank3.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_055000_rank4.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_055000_rank5.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_055000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_055000_rank0.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_055000_rank6.pt: 95%|█████████▌| 2.37GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.6GB / 22.2GB, 562MB/s New Data Upload : 87%|████████▋ | 5.52GB / 6.33GB, 204MB/s  ...eckpoints/model_055000.pt: 76%|███████▌ | 1.78GB / 2.35GB  ...nts/optim_055000_rank1.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_055000_rank2.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_055000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_055000_rank4.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_055000_rank5.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_055000_rank7.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_055000_rank0.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_055000_rank6.pt: 95%|█████████▌| 2.37GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.7GB / 22.2GB, 555MB/s New Data Upload : 88%|████████▊ | 5.56GB / 6.33GB, 204MB/s  ...eckpoints/model_055000.pt: 77%|███████▋ | 1.80GB / 2.35GB  ...nts/optim_055000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_055000_rank2.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_055000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_055000_rank4.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_055000_rank5.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_055000_rank7.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_055000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_055000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▎| 20.8GB / 22.2GB, 539MB/s New Data Upload : 88%|████████▊ | 5.59GB / 6.33GB, 203MB/s  ...eckpoints/model_055000.pt: 77%|███████▋ | 1.82GB / 2.35GB  ...nts/optim_055000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_055000_rank2.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_055000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_055000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_055000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_055000_rank7.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_055000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_055000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 20.9GB / 22.2GB, 528MB/s New Data Upload : 89%|████████▉ | 5.62GB / 6.33GB, 203MB/s  ...eckpoints/model_055000.pt: 78%|███████▊ | 1.83GB / 2.35GB  ...nts/optim_055000_rank1.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_055000_rank2.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_055000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_055000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_055000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_055000_rank7.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_055000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_055000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 20.9GB / 22.2GB, 513MB/s New Data Upload : 89%|████████▉ | 5.64GB / 6.33GB, 202MB/s  ...eckpoints/model_055000.pt: 79%|███████▉ | 1.85GB / 2.35GB  ...nts/optim_055000_rank1.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_055000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_055000_rank3.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_055000_rank4.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_055000_rank5.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_055000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_055000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_055000_rank6.pt: 96%|█████████▋| 2.39GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 21.0GB / 22.2GB, 503MB/s New Data Upload : 90%|████████▉ | 5.67GB / 6.33GB, 201MB/s  ...eckpoints/model_055000.pt: 80%|███████▉ | 1.87GB / 2.35GB  ...nts/optim_055000_rank1.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_055000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_055000_rank3.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_055000_rank4.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_055000_rank5.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_055000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_055000_rank0.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_055000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▍| 21.0GB / 22.2GB, 495MB/s New Data Upload : 90%|█████████ | 5.70GB / 6.33GB, 200MB/s  ...eckpoints/model_055000.pt: 80%|████████ | 1.89GB / 2.35GB  ...nts/optim_055000_rank1.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_055000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_055000_rank3.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_055000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_055000_rank5.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_055000_rank7.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_055000_rank0.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_055000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▍| 21.1GB / 22.2GB, 488MB/s New Data Upload : 90%|█████████ | 5.72GB / 6.33GB, 198MB/s  ...eckpoints/model_055000.pt: 81%|████████ | 1.90GB / 2.35GB  ...nts/optim_055000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_055000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_055000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_055000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_055000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_055000_rank0.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_055000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.2GB / 22.2GB, 485MB/s New Data Upload : 91%|█████████ | 5.76GB / 6.33GB, 198MB/s  ...eckpoints/model_055000.pt: 82%|████████▏ | 1.92GB / 2.35GB  ...nts/optim_055000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_055000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_055000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_055000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_055000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_055000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_055000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.2GB / 22.2GB, 480MB/s New Data Upload : 91%|█████████▏| 5.78GB / 6.33GB, 197MB/s  ...eckpoints/model_055000.pt: 83%|████████▎ | 1.94GB / 2.35GB  ...nts/optim_055000_rank1.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_055000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_055000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_055000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_055000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_055000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_055000_rank6.pt: 98%|█████████▊| 2.42GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 474MB/s New Data Upload : 92%|█████████▏| 5.81GB / 6.33GB, 197MB/s  ...eckpoints/model_055000.pt: 83%|████████▎ | 1.95GB / 2.35GB  ...nts/optim_055000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_055000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_055000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_055000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_055000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 467MB/s New Data Upload : 92%|█████████▏| 5.83GB / 6.33GB, 195MB/s  ...eckpoints/model_055000.pt: 84%|████████▍ | 1.97GB / 2.35GB  ...nts/optim_055000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_055000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_055000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▋| 21.4GB / 22.2GB, 459MB/s New Data Upload : 93%|█████████▎| 5.86GB / 6.33GB, 194MB/s  ...eckpoints/model_055000.pt: 85%|████████▍ | 1.99GB / 2.35GB  ...nts/optim_055000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank7.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_055000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▋| 21.4GB / 22.2GB, 455MB/s New Data Upload : 93%|█████████▎| 5.88GB / 6.33GB, 193MB/s  ...eckpoints/model_055000.pt: 85%|████████▌ | 2.01GB / 2.35GB  ...nts/optim_055000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 446MB/s New Data Upload : 93%|█████████▎| 5.91GB / 6.33GB, 191MB/s  ...eckpoints/model_055000.pt: 86%|████████▌ | 2.02GB / 2.35GB  ...nts/optim_055000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 442MB/s New Data Upload : 94%|█████████▎| 5.93GB / 6.33GB, 189MB/s  ...eckpoints/model_055000.pt: 87%|████████▋ | 2.04GB / 2.35GB  ...nts/optim_055000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 433MB/s New Data Upload : 94%|█████████▍| 5.95GB / 6.33GB, 187MB/s  ...eckpoints/model_055000.pt: 87%|████████▋ | 2.05GB / 2.35GB  ...nts/optim_055000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_055000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 426MB/s New Data Upload : 94%|█████████▍| 5.96GB / 6.33GB, 185MB/s  ...eckpoints/model_055000.pt: 88%|████████▊ | 2.07GB / 2.35GB  ...nts/optim_055000_rank1.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank6.pt: 98%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 418MB/s New Data Upload : 95%|█████████▍| 5.99GB / 6.33GB, 183MB/s  ...eckpoints/model_055000.pt: 89%|████████▉ | 2.09GB / 2.35GB  ...nts/optim_055000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank4.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.7GB / 22.2GB, 412MB/s New Data Upload : 95%|█████████▍| 6.01GB / 6.33GB, 182MB/s  ...eckpoints/model_055000.pt: 90%|████████▉ | 2.11GB / 2.35GB  ...nts/optim_055000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank2.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...nts/optim_055000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 402MB/s New Data Upload : 95%|█████████▌| 6.03GB / 6.33GB, 178MB/s  ...eckpoints/model_055000.pt: 90%|█████████ | 2.12GB / 2.35GB  ...nts/optim_055000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_055000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 398MB/s New Data Upload : 96%|█████████▌| 6.05GB / 6.33GB, 178MB/s  ...eckpoints/model_055000.pt: 91%|█████████ | 2.14GB / 2.35GB  ...nts/optim_055000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...nts/optim_055000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank7.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 390MB/s New Data Upload : 96%|█████████▌| 6.07GB / 6.33GB, 175MB/s  ...eckpoints/model_055000.pt: 91%|█████████▏| 2.15GB / 2.35GB  ...nts/optim_055000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 379MB/s New Data Upload : 96%|█████████▌| 6.08GB / 6.33GB, 171MB/s  ...eckpoints/model_055000.pt: 92%|█████████▏| 2.16GB / 2.35GB  ...nts/optim_055000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 374MB/s New Data Upload : 96%|█████████▋| 6.10GB / 6.33GB, 169MB/s  ...eckpoints/model_055000.pt: 93%|█████████▎| 2.17GB / 2.35GB  ...nts/optim_055000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 367MB/s New Data Upload : 97%|█████████▋| 6.11GB / 6.33GB, 166MB/s  ...eckpoints/model_055000.pt: 93%|█████████▎| 2.18GB / 2.35GB  ...nts/optim_055000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 356MB/s New Data Upload : 97%|█████████▋| 6.12GB / 6.33GB, 162MB/s  ...eckpoints/model_055000.pt: 93%|█████████▎| 2.19GB / 2.35GB  ...nts/optim_055000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 350MB/s New Data Upload : 97%|█████████▋| 6.13GB / 6.33GB, 160MB/s  ...eckpoints/model_055000.pt: 94%|█████████▍| 2.20GB / 2.35GB  ...nts/optim_055000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 339MB/s New Data Upload : 97%|█████████▋| 6.15GB / 6.33GB, 157MB/s  ...eckpoints/model_055000.pt: 94%|█████████▍| 2.21GB / 2.35GB  ...nts/optim_055000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 332MB/s New Data Upload : 97%|█████████▋| 6.16GB / 6.33GB, 154MB/s  ...eckpoints/model_055000.pt: 94%|█████████▍| 2.22GB / 2.35GB  ...nts/optim_055000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 321MB/s New Data Upload : 97%|█████████▋| 6.16GB / 6.33GB, 150MB/s  ...eckpoints/model_055000.pt: 95%|█████████▍| 2.22GB / 2.35GB  ...nts/optim_055000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 312MB/s New Data Upload : 98%|█████████▊| 6.17GB / 6.33GB, 146MB/s  ...eckpoints/model_055000.pt: 95%|█████████▌| 2.23GB / 2.35GB  ...nts/optim_055000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 305MB/s New Data Upload : 98%|█████████▊| 6.18GB / 6.33GB, 143MB/s  ...eckpoints/model_055000.pt: 95%|█████████▌| 2.24GB / 2.35GB  ...nts/optim_055000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 295MB/s New Data Upload : 98%|█████████▊| 6.19GB / 6.33GB, 138MB/s  ...eckpoints/model_055000.pt: 96%|█████████▌| 2.24GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 287MB/s New Data Upload : 98%|█████████▊| 6.20GB / 6.33GB, 135MB/s  ...eckpoints/model_055000.pt: 96%|█████████▌| 2.25GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 278MB/s New Data Upload : 98%|█████████▊| 6.21GB / 6.33GB, 131MB/s  ...eckpoints/model_055000.pt: 96%|█████████▌| 2.26GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 267MB/s New Data Upload : 98%|█████████▊| 6.21GB / 6.33GB, 126MB/s  ...eckpoints/model_055000.pt: 96%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 259MB/s New Data Upload : 98%|█████████▊| 6.22GB / 6.33GB, 123MB/s  ...eckpoints/model_055000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 249MB/s New Data Upload : 98%|█████████▊| 6.23GB / 6.33GB, 119MB/s  ...eckpoints/model_055000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 242MB/s New Data Upload : 99%|█████████▊| 6.24GB / 6.33GB, 116MB/s  ...eckpoints/model_055000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 232MB/s New Data Upload : 99%|█████████▊| 6.24GB / 6.33GB, 111MB/s  ...eckpoints/model_055000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 225MB/s New Data Upload : 99%|█████████▊| 6.25GB / 6.33GB, 108MB/s  ...eckpoints/model_055000.pt: 97%|█████████▋| 2.29GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 216MB/s New Data Upload : 99%|█████████▉| 6.25GB / 6.33GB, 104MB/s  ...eckpoints/model_055000.pt: 98%|█████████▊| 2.29GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 211MB/s New Data Upload : 99%|█████████▉| 6.26GB / 6.33GB, 101MB/s  ...eckpoints/model_055000.pt: 98%|█████████▊| 2.29GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 203MB/s New Data Upload : 99%|█████████▉| 6.27GB / 6.33GB, 97.6MB/s  ...eckpoints/model_055000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 193MB/s New Data Upload : 99%|█████████▉| 6.27GB / 6.33GB, 93.7MB/s  ...eckpoints/model_055000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 184MB/s New Data Upload : 99%|█████████▉| 6.27GB / 6.33GB, 90.1MB/s  ...eckpoints/model_055000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 176MB/s New Data Upload : 99%|█████████▉| 6.28GB / 6.33GB, 86.7MB/s  ...eckpoints/model_055000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 171MB/s New Data Upload : 99%|█████████▉| 6.29GB / 6.33GB, 84.4MB/s  ...eckpoints/model_055000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_055000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 158MB/s New Data Upload : 99%|█████████▉| 6.29GB / 6.33GB, 78.2MB/s  ...eckpoints/model_055000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 152MB/s New Data Upload : 99%|█████████▉| 6.30GB / 6.33GB, 75.8MB/s  ...eckpoints/model_055000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 144MB/s New Data Upload : 99%|█████████▉| 6.30GB / 6.33GB, 72.1MB/s  ...eckpoints/model_055000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 138MB/s New Data Upload : 100%|█████████▉| 6.30GB / 6.33GB, 70.0MB/s  ...eckpoints/model_055000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 132MB/s New Data Upload : 100%|█████████▉| 6.30GB / 6.33GB, 67.1MB/s  ...eckpoints/model_055000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 127MB/s New Data Upload : 100%|█████████▉| 6.31GB / 6.33GB, 65.1MB/s  ...eckpoints/model_055000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 122MB/s New Data Upload : 100%|█████████▉| 6.31GB / 6.33GB, 62.8MB/s  ...eckpoints/model_055000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 115MB/s New Data Upload : 100%|█████████▉| 6.31GB / 6.33GB, 60.2MB/s  ...eckpoints/model_055000.pt: 99%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 110MB/s New Data Upload : 100%|█████████▉| 6.32GB / 6.33GB, 58.2MB/s  ...eckpoints/model_055000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 101MB/s New Data Upload : 100%|█████████▉| 6.32GB / 6.33GB, 55.0MB/s  ...eckpoints/model_055000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 96.8MB/s New Data Upload : 100%|█████████▉| 6.32GB / 6.33GB, 53.0MB/s  ...eckpoints/model_055000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 90.0MB/s New Data Upload : 100%|█████████▉| 6.33GB / 6.33GB, 50.1MB/s  ...eckpoints/model_055000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 87.1MB/s New Data Upload : 100%|█████████▉| 6.33GB / 6.33GB, 48.5MB/s  ...eckpoints/model_055000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_055000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 77.0MB/s New Data Upload : 100%|█████████▉| 6.33GB / 6.33GB, 43.7MB/s  ...eckpoints/model_055000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_055000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_055000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 66.2MB/s New Data Upload : 100%|█████████▉| 6.33GB / 6.33GB, 37.0MB/s  ...eckpoints/model_055000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_055000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_055000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 56.2MB/s New Data Upload : 100%|██████████| 6.33GB / 6.33GB, 31.5MB/s  ...eckpoints/model_055000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_055000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_055000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_055000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_055000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_055000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_055000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_055000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_055000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_055000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_055000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_055000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 26.6MB/s New Data Upload : 100%|██████████| 6.33GB / 6.33GB, 15.7MB/s ...eckpoints/model_055000.pt: 100%|██████████| 2.35GB / 2.35GB ...nts/optim_055000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_055000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_055000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_055000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB ...nts/optim_055000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_055000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_055000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_055000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB [17:00:06] Step 55000 uploaded [18:00:06] Uploading step 60000... Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...eckpoints/model_060000.pt: 0%| | 585kB / 2.35GB  ...eckpoints/model_060000.pt: 0%| | 585kB / 2.35GB  Processing Files (0 / 1) : 0%| | 585kB / 22.2GB, 731kB/s New Data Upload : 0%| | 585kB / 536MB, 731kB/s  ...eckpoints/model_060000.pt: 0%| | 3.67MB / 2.35GB  Processing Files (0 / 1) : 0%| | 3.67MB / 22.2GB, 3.67MB/s ...eckpoints/model_060000.pt: 0%| | 4.25MB / 2.35GB  Processing Files (0 / 1) : 0%| | 4.25MB / 22.2GB, 3.54MB/s New Data Upload : 0%| | 1.17MB / 603MB, 975kB/s  ...eckpoints/model_060000.pt: 0%| | 7.18MB / 2.35GB  Processing Files (0 / 1) : 0%| | 7.18MB / 22.2GB, 5.12MB/s New Data Upload : 1%| | 4.10MB / 603MB, 2.92MB/s  ...eckpoints/model_060000.pt: 0%| | 11.3MB / 2.35GB  Processing Files (0 / 1) : 0%| | 11.3MB / 22.2GB, 7.04MB/s New Data Upload : 1%|▏ | 8.19MB / 603MB, 5.12MB/s  ...eckpoints/model_060000.pt: 1%| | 20.0MB / 2.35GB  Processing Files (0 / 1) : 0%| | 20.0MB / 22.2GB, 11.1MB/s New Data Upload : 3%|▎ | 17.0MB / 603MB, 9.42MB/s  ...eckpoints/model_060000.pt: 1%| | 24.1MB / 2.35GB  Processing Files (0 / 1) : 0%| | 24.1MB / 22.2GB, 12.1MB/s New Data Upload : 3%|▎ | 21.1MB / 603MB, 10.5MB/s  ...eckpoints/model_060000.pt: 1%| | 28.8MB / 2.35GB  Processing Files (0 / 1) : 0%| | 28.8MB / 22.2GB, 13.1MB/s New Data Upload : 4%|▍ | 25.7MB / 603MB, 11.7MB/s  ...eckpoints/model_060000.pt: 1%|▏ | 34.7MB / 2.35GB  Processing Files (0 / 1) : 0%| | 34.7MB / 22.2GB, 14.4MB/s New Data Upload : 5%|▌ | 31.6MB / 603MB, 13.2MB/s  ...eckpoints/model_060000.pt: 2%|▏ | 38.8MB / 2.35GB  Processing Files (0 / 1) : 0%| | 38.8MB / 22.2GB, 14.9MB/s New Data Upload : 6%|▌ | 35.7MB / 603MB, 13.7MB/s  ...eckpoints/model_060000.pt: 2%|▏ | 46.4MB / 2.35GB  Processing Files (0 / 1) : 0%| | 46.4MB / 22.2GB, 16.6MB/s New Data Upload : 7%|▋ | 43.3MB / 603MB, 15.5MB/s  ...eckpoints/model_060000.pt: 2%|▏ | 55.2MB / 2.35GB  Processing Files (0 / 1) : 0%| | 55.2MB / 22.2GB, 18.4MB/s New Data Upload : 9%|▊ | 52.1MB / 603MB, 17.4MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 59.2MB / 2.35GB  Processing Files (0 / 1) : 0%| | 59.2MB / 22.2GB, 18.5MB/s New Data Upload : 9%|▉ | 56.2MB / 603MB, 17.6MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 68.6MB / 2.35GB  Processing Files (0 / 1) : 0%| | 68.6MB / 22.2GB, 20.2MB/s New Data Upload : 11%|█ | 65.5MB / 603MB, 19.3MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 69.8MB / 2.35GB  Processing Files (0 / 1) : 0%| | 69.8MB / 22.2GB, 19.4MB/s New Data Upload : 11%|█ | 66.7MB / 603MB, 18.5MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 69.8MB / 2.35GB  ...eckpoints/model_060000.pt: 3%|▎ | 69.8MB / 2.35GB  ...eckpoints/model_060000.pt: 3%|▎ | 69.8MB / 2.35GB  ...eckpoints/model_060000.pt: 3%|▎ | 69.8MB / 2.35GB  ...nts/optim_060000_rank0.pt: 0%| | 385kB / 2.49GB  ...nts/optim_060000_rank1.pt: 0%| | 731kB / 2.49GB  ...nts/optim_060000_rank4.pt: 0%| | 731kB / 2.49GB  ...nts/optim_060000_rank5.pt: 0%| | 385kB / 2.49GB  ...nts/optim_060000_rank6.pt: 0%| | 4.15MB / 2.49GB  ...nts/optim_060000_rank7.pt: 0%| | 731kB / 2.49GB  ...nts/optim_060000_rank3.pt: 0%| | 1.17MB / 2.49GB  ...eckpoints/model_060000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 0%| | 385kB / 2.49GB  ...nts/optim_060000_rank1.pt: 0%| | 731kB / 2.49GB  ...nts/optim_060000_rank4.pt: 0%| | 731kB / 2.49GB  ...nts/optim_060000_rank5.pt: 0%| | 385kB / 2.49GB  ...nts/optim_060000_rank6.pt: 0%| | 4.15MB / 2.49GB  ...nts/optim_060000_rank7.pt: 0%| | 731kB / 2.49GB  ...nts/optim_060000_rank3.pt: 0%| | 1.17MB / 2.49GB  Processing Files (0 / 8) : 0%| | 78.3MB / 22.2GB, 17.0MB/s New Data Upload : 12%|█▏ | 72.2MB / 603MB, 15.7MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 0%| | 467kB / 2.49GB  ...nts/optim_060000_rank1.pt: 0%| | 1.02MB / 2.49GB  ...nts/optim_060000_rank4.pt: 0%| | 1.02MB / 2.49GB  ...nts/optim_060000_rank5.pt: 0%| | 467kB / 2.49GB  ...nts/optim_060000_rank6.pt: 0%| | 5.90MB / 2.49GB  ...nts/optim_060000_rank7.pt: 0%| | 1.02MB / 2.49GB  ...nts/optim_060000_rank3.pt: 0%| | 1.17MB / 2.49GB  Processing Files (0 / 8) : 0%| | 81.1MB / 22.2GB, 16.9MB/s New Data Upload : 10%|█ | 74.0MB / 738MB, 15.4MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 0%| | 794kB / 2.49GB  ...nts/optim_060000_rank1.pt: 0%| | 2.18MB / 2.49GB  ...nts/optim_060000_rank4.pt: 0%| | 2.18MB / 2.49GB  ...nts/optim_060000_rank5.pt: 0%| | 794kB / 2.49GB  ...nts/optim_060000_rank6.pt: 1%| | 12.9MB / 2.49GB  ...nts/optim_060000_rank7.pt: 0%| | 2.18MB / 2.49GB  ...nts/optim_060000_rank3.pt: 0%| | 1.17MB / 2.49GB  Processing Files (0 / 8) : 0%| | 92.3MB / 22.2GB, 18.5MB/s New Data Upload : 11%|█ | 81.0MB / 738MB, 16.2MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 0%| | 1.52MB / 2.49GB  ...nts/optim_060000_rank1.pt: 0%| | 3.80MB / 2.49GB  ...nts/optim_060000_rank4.pt: 0%| | 3.80MB / 2.49GB  ...nts/optim_060000_rank5.pt: 0%| | 1.52MB / 2.49GB  ...nts/optim_060000_rank6.pt: 1%| | 22.3MB / 2.49GB  ...nts/optim_060000_rank7.pt: 0%| | 3.80MB / 2.49GB  ...nts/optim_060000_rank3.pt: 0%| | 2.92MB / 2.49GB  Processing Files (0 / 8) : 0%| | 110MB / 22.2GB, 21.1MB/s New Data Upload : 12%|█▏ | 92.1MB / 738MB, 17.7MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 0%| | 2.91MB / 2.49GB  ...nts/optim_060000_rank1.pt: 0%| | 4.75MB / 2.49GB  ...nts/optim_060000_rank4.pt: 0%| | 4.75MB / 2.49GB  ...nts/optim_060000_rank5.pt: 0%| | 2.32MB / 2.49GB  ...nts/optim_060000_rank6.pt: 1%| | 27.2MB / 2.49GB  ...nts/optim_060000_rank7.pt: 0%| | 4.75MB / 2.49GB  ...nts/optim_060000_rank3.pt: 0%| | 6.44MB / 2.49GB  Processing Files (0 / 8) : 1%| | 123MB / 22.2GB, 22.8MB/s New Data Upload : 13%|█▎ | 101MB / 805MB, 18.7MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 0%| | 4.67MB / 2.49GB  ...nts/optim_060000_rank1.pt: 0%| | 6.43MB / 2.49GB  ...nts/optim_060000_rank4.pt: 0%| | 6.43MB / 2.49GB  ...nts/optim_060000_rank5.pt: 0%| | 3.51MB / 2.49GB  ...nts/optim_060000_rank6.pt: 1%|▏ | 36.2MB / 2.49GB  ...nts/optim_060000_rank7.pt: 0%| | 6.43MB / 2.49GB  ...nts/optim_060000_rank3.pt: 0%| | 11.1MB / 2.49GB  Processing Files (0 / 8) : 1%| | 145MB / 22.2GB, 25.9MB/s New Data Upload : 14%|█▍ | 115MB / 805MB, 20.5MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 0%| | 9.10MB / 2.49GB  ...nts/optim_060000_rank1.pt: 0%| | 8.34MB / 2.49GB  ...nts/optim_060000_rank4.pt: 0%| | 8.34MB / 2.49GB  ...nts/optim_060000_rank5.pt: 0%| | 5.09MB / 2.49GB  ...nts/optim_060000_rank6.pt: 2%|▏ | 45.9MB / 2.49GB  ...nts/optim_060000_rank7.pt: 0%| | 8.34MB / 2.49GB  ...nts/optim_060000_rank3.pt: 1%| | 17.6MB / 2.49GB  Processing Files (0 / 8) : 1%| | 173MB / 22.2GB, 29.8MB/s New Data Upload : 17%|█▋ | 134MB / 805MB, 23.0MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 1%| | 14.5MB / 2.49GB  ...nts/optim_060000_rank1.pt: 0%| | 9.05MB / 2.49GB  ...nts/optim_060000_rank4.pt: 0%| | 9.05MB / 2.49GB  ...nts/optim_060000_rank5.pt: 0%| | 9.87MB / 2.49GB  ...nts/optim_060000_rank6.pt: 2%|▏ | 46.5MB / 2.49GB  ...nts/optim_060000_rank7.pt: 0%| | 9.05MB / 2.49GB  ...nts/optim_060000_rank3.pt: 1%| | 26.0MB / 2.49GB  Processing Files (0 / 8) : 1%| | 194MB / 22.2GB, 32.3MB/s New Data Upload : 17%|█▋ | 146MB / 872MB, 24.4MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 1%| | 23.7MB / 2.49GB  ...nts/optim_060000_rank1.pt: 0%| | 10.7MB / 2.49GB  ...nts/optim_060000_rank4.pt: 0%| | 10.7MB / 2.49GB  ...nts/optim_060000_rank5.pt: 0%| | 12.2MB / 2.49GB  ...nts/optim_060000_rank6.pt: 2%|▏ | 51.7MB / 2.49GB  ...nts/optim_060000_rank7.pt: 0%| | 10.7MB / 2.49GB  ...nts/optim_060000_rank3.pt: 1%| | 31.0MB / 2.49GB  Processing Files (0 / 8) : 1%| | 221MB / 22.2GB, 35.6MB/s New Data Upload : 19%|█▉ | 165MB / 872MB, 26.6MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 1%|▏ | 33.6MB / 2.49GB  ...nts/optim_060000_rank1.pt: 1%| | 13.7MB / 2.49GB  ...nts/optim_060000_rank4.pt: 1%| | 13.7MB / 2.49GB  ...nts/optim_060000_rank5.pt: 1%| | 17.6MB / 2.49GB  ...nts/optim_060000_rank6.pt: 2%|▏ | 58.6MB / 2.49GB  ...nts/optim_060000_rank7.pt: 1%| | 16.4MB / 2.49GB  ...nts/optim_060000_rank3.pt: 2%|▏ | 40.4MB / 2.49GB  Processing Files (0 / 8) : 1%| | 264MB / 22.2GB, 41.2MB/s New Data Upload : 22%|██▏ | 191MB / 872MB, 29.9MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 2%|▏ | 40.2MB / 2.49GB  ...nts/optim_060000_rank1.pt: 1%| | 18.2MB / 2.49GB  ...nts/optim_060000_rank4.pt: 1%| | 18.2MB / 2.49GB  ...nts/optim_060000_rank5.pt: 1%| | 26.6MB / 2.49GB  ...nts/optim_060000_rank6.pt: 3%|▎ | 67.2MB / 2.49GB  ...nts/optim_060000_rank7.pt: 1%| | 21.5MB / 2.49GB  ...nts/optim_060000_rank3.pt: 2%|▏ | 50.3MB / 2.49GB  Processing Files (0 / 8) : 1%|▏ | 312MB / 22.2GB, 47.3MB/s New Data Upload : 23%|██▎ | 219MB / 939MB, 33.3MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 2%|▏ | 44.8MB / 2.49GB  ...nts/optim_060000_rank1.pt: 1%| | 19.3MB / 2.49GB  ...nts/optim_060000_rank4.pt: 1%| | 19.3MB / 2.49GB  ...nts/optim_060000_rank5.pt: 1%| | 28.3MB / 2.49GB  ...nts/optim_060000_rank6.pt: 3%|▎ | 71.4MB / 2.49GB  ...nts/optim_060000_rank7.pt: 1%| | 23.7MB / 2.49GB  ...nts/optim_060000_rank3.pt: 2%|▏ | 58.2MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 335MB / 22.2GB, 49.3MB/s New Data Upload : 25%|██▌ | 235MB / 939MB, 34.5MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 2%|▏ | 49.1MB / 2.49GB  ...nts/optim_060000_rank1.pt: 1%| | 21.4MB / 2.49GB  ...nts/optim_060000_rank4.pt: 1%| | 24.5MB / 2.49GB  ...nts/optim_060000_rank5.pt: 1%|▏ | 33.3MB / 2.49GB  ...nts/optim_060000_rank6.pt: 3%|▎ | 73.1MB / 2.49GB  ...nts/optim_060000_rank7.pt: 1%| | 30.0MB / 2.49GB  ...nts/optim_060000_rank3.pt: 3%|▎ | 62.2MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 364MB / 22.2GB, 51.9MB/s New Data Upload : 26%|██▋ | 249MB / 939MB, 35.5MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 2%|▏ | 57.8MB / 2.49GB  ...nts/optim_060000_rank1.pt: 1%| | 24.1MB / 2.49GB  ...nts/optim_060000_rank4.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_060000_rank5.pt: 2%|▏ | 39.7MB / 2.49GB  ...nts/optim_060000_rank6.pt: 3%|▎ | 75.9MB / 2.49GB  ...nts/optim_060000_rank7.pt: 2%|▏ | 38.0MB / 2.49GB  ...nts/optim_060000_rank3.pt: 3%|▎ | 69.9MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 402MB / 22.2GB, 55.9MB/s New Data Upload : 29%|██▉ | 271MB / 939MB, 37.7MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 3%|▎ | 62.4MB / 2.49GB  ...nts/optim_060000_rank1.pt: 1%| | 26.3MB / 2.49GB  ...nts/optim_060000_rank4.pt: 1%| | 30.0MB / 2.49GB  ...nts/optim_060000_rank5.pt: 2%|▏ | 44.5MB / 2.49GB  ...nts/optim_060000_rank6.pt: 3%|▎ | 76.7MB / 2.49GB  ...nts/optim_060000_rank7.pt: 2%|▏ | 40.0MB / 2.49GB  ...nts/optim_060000_rank3.pt: 3%|▎ | 73.6MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 424MB / 22.2GB, 57.2MB/s New Data Upload : 30%|███ | 284MB / 939MB, 38.3MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 3%|▎ | 65.7MB / 2.49GB  ...nts/optim_060000_rank1.pt: 1%|▏ | 32.4MB / 2.49GB  ...nts/optim_060000_rank4.pt: 1%|▏ | 36.2MB / 2.49GB  ...nts/optim_060000_rank5.pt: 2%|▏ | 51.3MB / 2.49GB  ...nts/optim_060000_rank6.pt: 3%|▎ | 78.0MB / 2.49GB  ...nts/optim_060000_rank7.pt: 2%|▏ | 42.9MB / 2.49GB  ...nts/optim_060000_rank3.pt: 3%|▎ | 74.9MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 451MB / 22.2GB, 59.4MB/s New Data Upload : 29%|██▉ | 297MB / 1.01GB, 39.0MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 3%|▎ | 71.5MB / 2.49GB  ...nts/optim_060000_rank1.pt: 1%|▏ | 36.1MB / 2.49GB  ...nts/optim_060000_rank4.pt: 2%|▏ | 40.8MB / 2.49GB  ...nts/optim_060000_rank5.pt: 2%|▏ | 56.7MB / 2.49GB  ...nts/optim_060000_rank6.pt: 3%|▎ | 79.9MB / 2.49GB  ...nts/optim_060000_rank7.pt: 2%|▏ | 45.4MB / 2.49GB  ...nts/optim_060000_rank3.pt: 3%|▎ | 76.3MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 477MB / 22.2GB, 61.1MB/s New Data Upload : 31%|███ | 310MB / 1.01GB, 39.8MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.6MB / 2.35GB  ...nts/optim_060000_rank0.pt: 3%|▎ | 75.2MB / 2.49GB  ...nts/optim_060000_rank1.pt: 2%|▏ | 41.5MB / 2.49GB  ...nts/optim_060000_rank4.pt: 2%|▏ | 44.5MB / 2.49GB  ...nts/optim_060000_rank5.pt: 2%|▏ | 61.5MB / 2.49GB  ...nts/optim_060000_rank6.pt: 3%|▎ | 82.0MB / 2.49GB  ...nts/optim_060000_rank7.pt: 2%|▏ | 47.5MB / 2.49GB  ...nts/optim_060000_rank3.pt: 3%|▎ | 86.3MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 509MB / 22.2GB, 63.6MB/s New Data Upload : 29%|██▊ | 327MB / 1.14GB, 40.9MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.6MB / 2.35GB  ...nts/optim_060000_rank0.pt: 3%|▎ | 80.8MB / 2.49GB  ...nts/optim_060000_rank1.pt: 2%|▏ | 46.0MB / 2.49GB  ...nts/optim_060000_rank4.pt: 2%|▏ | 48.9MB / 2.49GB  ...nts/optim_060000_rank5.pt: 3%|▎ | 66.7MB / 2.49GB  ...nts/optim_060000_rank6.pt: 3%|▎ | 83.8MB / 2.49GB  ...nts/optim_060000_rank7.pt: 2%|▏ | 49.8MB / 2.49GB  ...nts/optim_060000_rank3.pt: 4%|▎ | 90.4MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 537MB / 22.2GB, 65.5MB/s New Data Upload : 30%|███ | 343MB / 1.14GB, 41.8MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 70.6MB / 2.35GB  ...nts/optim_060000_rank0.pt: 4%|▍ | 96.0MB / 2.49GB  ...nts/optim_060000_rank1.pt: 2%|▏ | 57.2MB / 2.49GB  ...nts/optim_060000_rank4.pt: 2%|▏ | 59.2MB / 2.49GB  ...nts/optim_060000_rank5.pt: 3%|▎ | 76.3MB / 2.49GB  ...nts/optim_060000_rank6.pt: 4%|▍ | 95.1MB / 2.49GB  ...nts/optim_060000_rank7.pt: 2%|▏ | 60.3MB / 2.49GB  ...nts/optim_060000_rank3.pt: 4%|▍ | 101MB / 2.49GB  Processing Files (0 / 8) : 3%|▎ | 616MB / 22.2GB, 73.3MB/s New Data Upload : 31%|███ | 373MB / 1.21GB, 44.5MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 74.1MB / 2.35GB  ...nts/optim_060000_rank0.pt: 4%|▍ | 98.8MB / 2.49GB  ...nts/optim_060000_rank1.pt: 3%|▎ | 63.6MB / 2.49GB  ...nts/optim_060000_rank4.pt: 3%|▎ | 64.4MB / 2.49GB  ...nts/optim_060000_rank5.pt: 3%|▎ | 82.0MB / 2.49GB  ...nts/optim_060000_rank6.pt: 4%|▍ | 97.6MB / 2.49GB  ...nts/optim_060000_rank7.pt: 3%|▎ | 62.8MB / 2.49GB  ...nts/optim_060000_rank3.pt: 4%|▍ | 110MB / 2.49GB  Processing Files (0 / 8) : 3%|▎ | 653MB / 22.2GB, 76.0MB/s New Data Upload : 33%|███▎ | 396MB / 1.21GB, 46.0MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 76.9MB / 2.35GB  ...nts/optim_060000_rank0.pt: 4%|▍ | 105MB / 2.49GB  ...nts/optim_060000_rank1.pt: 3%|▎ | 74.3MB / 2.49GB  ...nts/optim_060000_rank4.pt: 3%|▎ | 72.7MB / 2.49GB  ...nts/optim_060000_rank5.pt: 4%|▎ | 91.7MB / 2.49GB  ...nts/optim_060000_rank6.pt: 4%|▍ | 105MB / 2.49GB  ...nts/optim_060000_rank7.pt: 3%|▎ | 72.3MB / 2.49GB  ...nts/optim_060000_rank3.pt: 5%|▍ | 117MB / 2.49GB  Processing Files (0 / 8) : 3%|▎ | 715MB / 22.2GB, 81.2MB/s New Data Upload : 35%|███▍ | 419MB / 1.21GB, 47.6MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 78.6MB / 2.35GB  ...nts/optim_060000_rank0.pt: 5%|▍ | 115MB / 2.49GB  ...nts/optim_060000_rank1.pt: 4%|▎ | 88.0MB / 2.49GB  ...nts/optim_060000_rank4.pt: 3%|▎ | 84.1MB / 2.49GB  ...nts/optim_060000_rank5.pt: 4%|▍ | 105MB / 2.49GB  ...nts/optim_060000_rank6.pt: 5%|▍ | 115MB / 2.49GB  ...nts/optim_060000_rank7.pt: 3%|▎ | 86.4MB / 2.49GB  ...nts/optim_060000_rank3.pt: 5%|▌ | 131MB / 2.49GB  Processing Files (0 / 8) : 4%|▎ | 804MB / 22.2GB, 89.3MB/s New Data Upload : 37%|███▋ | 452MB / 1.21GB, 50.2MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 80.9MB / 2.35GB  ...nts/optim_060000_rank0.pt: 5%|▌ | 126MB / 2.49GB  ...nts/optim_060000_rank1.pt: 4%|▍ | 102MB / 2.49GB  ...nts/optim_060000_rank4.pt: 4%|▍ | 96.9MB / 2.49GB  ...nts/optim_060000_rank5.pt: 5%|▍ | 120MB / 2.49GB  ...nts/optim_060000_rank6.pt: 5%|▌ | 126MB / 2.49GB  ...nts/optim_060000_rank7.pt: 4%|▍ | 103MB / 2.49GB  ...nts/optim_060000_rank3.pt: 6%|▌ | 143MB / 2.49GB  Processing Files (0 / 8) : 4%|▍ | 898MB / 22.2GB, 97.6MB/s New Data Upload : 36%|███▌ | 483MB / 1.34GB, 52.5MB/s  ...eckpoints/model_060000.pt: 3%|▎ | 80.9MB / 2.35GB  ...nts/optim_060000_rank0.pt: 6%|▌ | 138MB / 2.49GB  ...nts/optim_060000_rank1.pt: 5%|▍ | 117MB / 2.49GB  ...nts/optim_060000_rank4.pt: 4%|▍ | 110MB / 2.49GB  ...nts/optim_060000_rank5.pt: 5%|▌ | 136MB / 2.49GB  ...nts/optim_060000_rank6.pt: 6%|▌ | 138MB / 2.49GB  ...nts/optim_060000_rank7.pt: 5%|▍ | 122MB / 2.49GB  ...nts/optim_060000_rank3.pt: 6%|▋ | 157MB / 2.49GB  Processing Files (0 / 8) : 4%|▍ | 999MB / 22.2GB, 106MB/s New Data Upload : 38%|███▊ | 514MB / 1.34GB, 54.7MB/s  ...eckpoints/model_060000.pt: 4%|▎ | 83.2MB / 2.35GB  ...nts/optim_060000_rank0.pt: 6%|▋ | 157MB / 2.49GB  ...nts/optim_060000_rank1.pt: 6%|▌ | 138MB / 2.49GB  ...nts/optim_060000_rank4.pt: 5%|▌ | 131MB / 2.49GB  ...nts/optim_060000_rank5.pt: 6%|▋ | 156MB / 2.49GB  ...nts/optim_060000_rank6.pt: 6%|▋ | 156MB / 2.49GB  ...nts/optim_060000_rank7.pt: 6%|▌ | 144MB / 2.49GB  ...nts/optim_060000_rank3.pt: 7%|▋ | 178MB / 2.49GB  Processing Files (0 / 8) : 5%|▌ | 1.14GB / 22.2GB, 119MB/s New Data Upload : 39%|███▉ | 554MB / 1.41GB, 57.7MB/s  ...eckpoints/model_060000.pt: 4%|▎ | 84.9MB / 2.35GB  ...nts/optim_060000_rank0.pt: 7%|▋ | 170MB / 2.49GB  ...nts/optim_060000_rank1.pt: 6%|▌ | 153MB / 2.49GB  ...nts/optim_060000_rank4.pt: 6%|▌ | 143MB / 2.49GB  ...nts/optim_060000_rank5.pt: 7%|▋ | 169MB / 2.49GB  ...nts/optim_060000_rank6.pt: 7%|▋ | 169MB / 2.49GB  ...nts/optim_060000_rank7.pt: 6%|▋ | 157MB / 2.49GB  ...nts/optim_060000_rank3.pt: 8%|▊ | 196MB / 2.49GB  Processing Files (0 / 8) : 6%|▌ | 1.24GB / 22.2GB, 127MB/s New Data Upload : 41%|████ | 579MB / 1.41GB, 59.0MB/s  ...eckpoints/model_060000.pt: 4%|▎ | 87.2MB / 2.35GB  ...nts/optim_060000_rank0.pt: 8%|▊ | 187MB / 2.49GB  ...nts/optim_060000_rank1.pt: 7%|▋ | 171MB / 2.49GB  ...nts/optim_060000_rank4.pt: 6%|▋ | 161MB / 2.49GB  ...nts/optim_060000_rank5.pt: 7%|▋ | 185MB / 2.49GB  ...nts/optim_060000_rank6.pt: 7%|▋ | 184MB / 2.49GB  ...nts/optim_060000_rank7.pt: 7%|▋ | 176MB / 2.49GB  ...nts/optim_060000_rank3.pt: 8%|▊ | 211MB / 2.49GB  Processing Files (0 / 8) : 6%|▌ | 1.36GB / 22.2GB, 136MB/s New Data Upload : 43%|████▎ | 610MB / 1.41GB, 61.0MB/s  ...eckpoints/model_060000.pt: 4%|▍ | 88.9MB / 2.35GB  ...nts/optim_060000_rank0.pt: 8%|▊ | 204MB / 2.49GB  ...nts/optim_060000_rank1.pt: 8%|▊ | 191MB / 2.49GB  ...nts/optim_060000_rank4.pt: 7%|▋ | 179MB / 2.49GB  ...nts/optim_060000_rank5.pt: 8%|▊ | 202MB / 2.49GB  ...nts/optim_060000_rank6.pt: 8%|▊ | 201MB / 2.49GB  ...nts/optim_060000_rank7.pt: 8%|▊ | 192MB / 2.49GB  ...nts/optim_060000_rank3.pt: 9%|▉ | 231MB / 2.49GB  Processing Files (0 / 8) : 7%|▋ | 1.49GB / 22.2GB, 146MB/s New Data Upload : 45%|████▌ | 639MB / 1.41GB, 62.6MB/s  ...eckpoints/model_060000.pt: 4%|▍ | 91.8MB / 2.35GB  ...nts/optim_060000_rank0.pt: 9%|▉ | 221MB / 2.49GB  ...nts/optim_060000_rank1.pt: 8%|▊ | 210MB / 2.49GB  ...nts/optim_060000_rank4.pt: 8%|▊ | 196MB / 2.49GB  ...nts/optim_060000_rank5.pt: 9%|▉ | 218MB / 2.49GB  ...nts/optim_060000_rank6.pt: 9%|▉ | 218MB / 2.49GB  ...nts/optim_060000_rank7.pt: 9%|▊ | 212MB / 2.49GB  ...nts/optim_060000_rank3.pt: 10%|█ | 249MB / 2.49GB  Processing Files (0 / 8) : 7%|▋ | 1.62GB / 22.2GB, 158MB/s New Data Upload : 45%|████▌ | 671MB / 1.48GB, 65.8MB/s  ...eckpoints/model_060000.pt: 4%|▍ | 94.6MB / 2.35GB  ...nts/optim_060000_rank0.pt: 10%|▉ | 246MB / 2.49GB  ...nts/optim_060000_rank1.pt: 10%|▉ | 237MB / 2.49GB  ...nts/optim_060000_rank4.pt: 9%|▉ | 221MB / 2.49GB  ...nts/optim_060000_rank5.pt: 10%|▉ | 243MB / 2.49GB  ...nts/optim_060000_rank6.pt: 10%|▉ | 242MB / 2.49GB  ...nts/optim_060000_rank7.pt: 10%|▉ | 239MB / 2.49GB  ...nts/optim_060000_rank3.pt: 11%|█ | 274MB / 2.49GB  Processing Files (0 / 8) : 8%|▊ | 1.80GB / 22.2GB, 176MB/s New Data Upload : 46%|████▌ | 711MB / 1.54GB, 69.7MB/s  ...eckpoints/model_060000.pt: 4%|▍ | 96.9MB / 2.35GB  ...nts/optim_060000_rank0.pt: 11%|█ | 265MB / 2.49GB  ...nts/optim_060000_rank1.pt: 10%|█ | 257MB / 2.49GB  ...nts/optim_060000_rank4.pt: 10%|▉ | 241MB / 2.49GB  ...nts/optim_060000_rank5.pt: 11%|█ | 262MB / 2.49GB  ...nts/optim_060000_rank6.pt: 10%|█ | 261MB / 2.49GB  ...nts/optim_060000_rank7.pt: 10%|█ | 260MB / 2.49GB  ...nts/optim_060000_rank3.pt: 12%|█▏ | 294MB / 2.49GB  Processing Files (0 / 8) : 9%|▊ | 1.94GB / 22.2GB, 190MB/s New Data Upload : 44%|████▍ | 740MB / 1.68GB, 72.6MB/s  ...eckpoints/model_060000.pt: 4%|▍ | 99.2MB / 2.35GB  ...nts/optim_060000_rank0.pt: 11%|█▏ | 285MB / 2.49GB  ...nts/optim_060000_rank1.pt: 11%|█ | 279MB / 2.49GB  ...nts/optim_060000_rank4.pt: 10%|█ | 261MB / 2.49GB  ...nts/optim_060000_rank5.pt: 11%|█▏ | 281MB / 2.49GB  ...nts/optim_060000_rank6.pt: 11%|█▏ | 280MB / 2.49GB  ...nts/optim_060000_rank7.pt: 11%|█▏ | 281MB / 2.49GB  ...nts/optim_060000_rank3.pt: 13%|█▎ | 313MB / 2.49GB  Processing Files (0 / 8) : 9%|▉ | 2.08GB / 22.2GB, 204MB/s New Data Upload : 46%|████▌ | 773MB / 1.68GB, 75.8MB/s  ...eckpoints/model_060000.pt: 4%|▍ | 103MB / 2.35GB  ...nts/optim_060000_rank0.pt: 13%|█▎ | 311MB / 2.49GB  ...nts/optim_060000_rank1.pt: 12%|█▏ | 305MB / 2.49GB  ...nts/optim_060000_rank4.pt: 12%|█▏ | 287MB / 2.49GB  ...nts/optim_060000_rank5.pt: 12%|█▏ | 305MB / 2.49GB  ...nts/optim_060000_rank6.pt: 12%|█▏ | 305MB / 2.49GB  ...nts/optim_060000_rank7.pt: 12%|█▏ | 308MB / 2.49GB  ...nts/optim_060000_rank3.pt: 14%|█▎ | 337MB / 2.49GB  Processing Files (0 / 8) : 10%|█ | 2.26GB / 22.2GB, 221MB/s New Data Upload : 46%|████▋ | 808MB / 1.74GB, 79.1MB/s  ...eckpoints/model_060000.pt: 5%|▍ | 109MB / 2.35GB  ...nts/optim_060000_rank0.pt: 13%|█▎ | 327MB / 2.49GB  ...nts/optim_060000_rank1.pt: 13%|█▎ | 321MB / 2.49GB  ...nts/optim_060000_rank4.pt: 12%|█▏ | 304MB / 2.49GB  ...nts/optim_060000_rank5.pt: 13%|█▎ | 321MB / 2.49GB  ...nts/optim_060000_rank6.pt: 13%|█▎ | 321MB / 2.49GB  ...nts/optim_060000_rank7.pt: 13%|█▎ | 324MB / 2.49GB  ...nts/optim_060000_rank3.pt: 14%|█▍ | 350MB / 2.49GB  Processing Files (0 / 8) : 11%|█ | 2.38GB / 22.2GB, 233MB/s New Data Upload : 48%|████▊ | 832MB / 1.74GB, 81.5MB/s  ...eckpoints/model_060000.pt: 5%|▍ | 113MB / 2.35GB  ...nts/optim_060000_rank0.pt: 14%|█▍ | 348MB / 2.49GB  ...nts/optim_060000_rank1.pt: 14%|█▍ | 342MB / 2.49GB  ...nts/optim_060000_rank4.pt: 13%|█▎ | 326MB / 2.49GB  ...nts/optim_060000_rank5.pt: 14%|█▎ | 341MB / 2.49GB  ...nts/optim_060000_rank6.pt: 14%|█▎ | 341MB / 2.49GB  ...nts/optim_060000_rank7.pt: 14%|█▍ | 344MB / 2.49GB  ...nts/optim_060000_rank3.pt: 15%|█▍ | 369MB / 2.49GB  Processing Files (0 / 8) : 11%|█▏ | 2.52GB / 22.2GB, 247MB/s New Data Upload : 48%|████▊ | 860MB / 1.81GB, 84.0MB/s  ...eckpoints/model_060000.pt: 5%|▌ | 121MB / 2.35GB  ...nts/optim_060000_rank0.pt: 15%|█▍ | 369MB / 2.49GB  ...nts/optim_060000_rank1.pt: 15%|█▍ | 361MB / 2.49GB  ...nts/optim_060000_rank4.pt: 14%|█▍ | 348MB / 2.49GB  ...nts/optim_060000_rank5.pt: 14%|█▍ | 360MB / 2.49GB  ...nts/optim_060000_rank6.pt: 15%|█▍ | 361MB / 2.49GB  ...nts/optim_060000_rank7.pt: 15%|█▍ | 364MB / 2.49GB  ...nts/optim_060000_rank3.pt: 16%|█▌ | 386MB / 2.49GB  Processing Files (0 / 8) : 12%|█▏ | 2.67GB / 22.2GB, 260MB/s New Data Upload : 49%|████▉ | 892MB / 1.81GB, 86.7MB/s  ...eckpoints/model_060000.pt: 5%|▌ | 125MB / 2.35GB  ...nts/optim_060000_rank0.pt: 16%|█▌ | 390MB / 2.49GB  ...nts/optim_060000_rank1.pt: 15%|█▌ | 381MB / 2.49GB  ...nts/optim_060000_rank4.pt: 15%|█▍ | 369MB / 2.49GB  ...nts/optim_060000_rank5.pt: 15%|█▌ | 378MB / 2.49GB  ...nts/optim_060000_rank6.pt: 15%|█▌ | 380MB / 2.49GB  ...nts/optim_060000_rank7.pt: 15%|█▌ | 383MB / 2.49GB  ...nts/optim_060000_rank3.pt: 16%|█▌ | 403MB / 2.49GB  Processing Files (0 / 8) : 13%|█▎ | 2.81GB / 22.2GB, 273MB/s New Data Upload : 51%|█████ | 920MB / 1.81GB, 88.5MB/s  ...eckpoints/model_060000.pt: 6%|▌ | 134MB / 2.35GB  ...nts/optim_060000_rank0.pt: 17%|█▋ | 419MB / 2.49GB  ...nts/optim_060000_rank1.pt: 16%|█▋ | 407MB / 2.49GB  ...nts/optim_060000_rank4.pt: 16%|█▌ | 396MB / 2.49GB  ...nts/optim_060000_rank5.pt: 16%|█▌ | 403MB / 2.49GB  ...nts/optim_060000_rank6.pt: 16%|█▋ | 405MB / 2.49GB  ...nts/optim_060000_rank7.pt: 16%|█▋ | 409MB / 2.49GB  ...nts/optim_060000_rank3.pt: 17%|█▋ | 426MB / 2.49GB  Processing Files (0 / 8) : 13%|█▎ | 3.00GB / 22.2GB, 292MB/s New Data Upload : 51%|█████▏ | 965MB / 1.88GB, 92.5MB/s  ...eckpoints/model_060000.pt: 6%|▌ | 147MB / 2.35GB  ...nts/optim_060000_rank0.pt: 18%|█▊ | 445MB / 2.49GB  ...nts/optim_060000_rank1.pt: 17%|█▋ | 433MB / 2.49GB  ...nts/optim_060000_rank4.pt: 17%|█▋ | 424MB / 2.49GB  ...nts/optim_060000_rank5.pt: 17%|█▋ | 428MB / 2.49GB  ...nts/optim_060000_rank6.pt: 17%|█▋ | 429MB / 2.49GB  ...nts/optim_060000_rank7.pt: 17%|█▋ | 433MB / 2.49GB  ...nts/optim_060000_rank3.pt: 18%|█▊ | 449MB / 2.49GB  Processing Files (0 / 8) : 14%|█▍ | 3.19GB / 22.2GB, 310MB/s New Data Upload : 52%|█████▏ | 1.01GB / 1.94GB, 96.5MB/s  ...eckpoints/model_060000.pt: 7%|▋ | 153MB / 2.35GB  ...nts/optim_060000_rank0.pt: 19%|█▉ | 471MB / 2.49GB  ...nts/optim_060000_rank1.pt: 18%|█▊ | 459MB / 2.49GB  ...nts/optim_060000_rank4.pt: 18%|█▊ | 452MB / 2.49GB  ...nts/optim_060000_rank5.pt: 18%|█▊ | 452MB / 2.49GB  ...nts/optim_060000_rank6.pt: 18%|█▊ | 453MB / 2.49GB  ...nts/optim_060000_rank7.pt: 18%|█▊ | 457MB / 2.49GB  ...nts/optim_060000_rank3.pt: 19%|█▉ | 470MB / 2.49GB  Processing Files (0 / 8) : 15%|█▌ | 3.37GB / 22.2GB, 327MB/s New Data Upload : 52%|█████▏ | 1.05GB / 2.01GB, 100MB/s  ...eckpoints/model_060000.pt: 7%|▋ | 162MB / 2.35GB  ...nts/optim_060000_rank0.pt: 20%|█▉ | 493MB / 2.49GB  ...nts/optim_060000_rank1.pt: 19%|█▉ | 479MB / 2.49GB  ...nts/optim_060000_rank4.pt: 19%|█▉ | 474MB / 2.49GB  ...nts/optim_060000_rank5.pt: 19%|█▉ | 473MB / 2.49GB  ...nts/optim_060000_rank6.pt: 19%|█▉ | 475MB / 2.49GB  ...nts/optim_060000_rank7.pt: 19%|█▉ | 479MB / 2.49GB  ...nts/optim_060000_rank3.pt: 20%|█▉ | 490MB / 2.49GB  Processing Files (0 / 8) : 16%|█▌ | 3.52GB / 22.2GB, 342MB/s New Data Upload : 52%|█████▏ | 1.09GB / 2.08GB, 103MB/s  ...eckpoints/model_060000.pt: 7%|▋ | 168MB / 2.35GB  ...nts/optim_060000_rank0.pt: 21%|██ | 514MB / 2.49GB  ...nts/optim_060000_rank1.pt: 20%|█▉ | 497MB / 2.49GB  ...nts/optim_060000_rank4.pt: 20%|█▉ | 494MB / 2.49GB  ...nts/optim_060000_rank5.pt: 20%|█▉ | 490MB / 2.49GB  ...nts/optim_060000_rank6.pt: 20%|█▉ | 493MB / 2.49GB  ...nts/optim_060000_rank7.pt: 20%|█▉ | 497MB / 2.49GB  ...nts/optim_060000_rank3.pt: 20%|██ | 506MB / 2.49GB  Processing Files (0 / 8) : 16%|█▋ | 3.66GB / 22.2GB, 354MB/s New Data Upload : 52%|█████▏ | 1.12GB / 2.15GB, 105MB/s  ...eckpoints/model_060000.pt: 7%|▋ | 176MB / 2.35GB  ...nts/optim_060000_rank0.pt: 22%|██▏ | 548MB / 2.49GB  ...nts/optim_060000_rank1.pt: 21%|██▏ | 530MB / 2.49GB  ...nts/optim_060000_rank4.pt: 21%|██▏ | 529MB / 2.49GB  ...nts/optim_060000_rank5.pt: 21%|██ | 523MB / 2.49GB  ...nts/optim_060000_rank6.pt: 21%|██ | 526MB / 2.49GB  ...nts/optim_060000_rank7.pt: 21%|██▏ | 532MB / 2.49GB  ...nts/optim_060000_rank3.pt: 22%|██▏ | 537MB / 2.49GB  Processing Files (0 / 8) : 18%|█▊ | 3.90GB / 22.2GB, 377MB/s New Data Upload : 54%|█████▍ | 1.17GB / 2.15GB, 110MB/s  ...eckpoints/model_060000.pt: 8%|▊ | 185MB / 2.35GB  ...nts/optim_060000_rank0.pt: 23%|██▎ | 581MB / 2.49GB  ...nts/optim_060000_rank1.pt: 22%|██▏ | 558MB / 2.49GB  ...nts/optim_060000_rank4.pt: 22%|██▏ | 558MB / 2.49GB  ...nts/optim_060000_rank5.pt: 22%|██▏ | 552MB / 2.49GB  ...nts/optim_060000_rank6.pt: 22%|██▏ | 556MB / 2.49GB  ...nts/optim_060000_rank7.pt: 23%|██▎ | 562MB / 2.49GB  ...nts/optim_060000_rank3.pt: 23%|██▎ | 565MB / 2.49GB  Processing Files (0 / 8) : 19%|█▊ | 4.12GB / 22.2GB, 398MB/s New Data Upload : 55%|█████▌ | 1.22GB / 2.21GB, 114MB/s  ...eckpoints/model_060000.pt: 8%|▊ | 187MB / 2.35GB  ...nts/optim_060000_rank0.pt: 25%|██▍ | 609MB / 2.49GB  ...nts/optim_060000_rank1.pt: 23%|██▎ | 584MB / 2.49GB  ...nts/optim_060000_rank4.pt: 23%|██▎ | 583MB / 2.49GB  ...nts/optim_060000_rank5.pt: 23%|██▎ | 579MB / 2.49GB  ...nts/optim_060000_rank6.pt: 23%|██▎ | 583MB / 2.49GB  ...nts/optim_060000_rank7.pt: 24%|██▎ | 589MB / 2.49GB  ...nts/optim_060000_rank3.pt: 24%|██▍ | 591MB / 2.49GB  Processing Files (0 / 8) : 19%|█▉ | 4.31GB / 22.2GB, 415MB/s New Data Upload : 55%|█████▌ | 1.26GB / 2.28GB, 117MB/s  ...eckpoints/model_060000.pt: 8%|▊ | 199MB / 2.35GB  ...nts/optim_060000_rank0.pt: 26%|██▌ | 641MB / 2.49GB  ...nts/optim_060000_rank1.pt: 25%|██▍ | 610MB / 2.49GB  ...nts/optim_060000_rank4.pt: 24%|██▍ | 608MB / 2.49GB  ...nts/optim_060000_rank5.pt: 24%|██▍ | 608MB / 2.49GB  ...nts/optim_060000_rank6.pt: 25%|██▍ | 612MB / 2.49GB  ...nts/optim_060000_rank7.pt: 25%|██▍ | 618MB / 2.49GB  ...nts/optim_060000_rank3.pt: 25%|██▍ | 617MB / 2.49GB  Processing Files (0 / 8) : 20%|██ | 4.51GB / 22.2GB, 436MB/s New Data Upload : 57%|█████▋ | 1.30GB / 2.28GB, 121MB/s  ...eckpoints/model_060000.pt: 9%|▊ | 204MB / 2.35GB  ...nts/optim_060000_rank0.pt: 27%|██▋ | 669MB / 2.49GB  ...nts/optim_060000_rank1.pt: 26%|██▌ | 637MB / 2.49GB  ...nts/optim_060000_rank4.pt: 26%|██▌ | 635MB / 2.49GB  ...nts/optim_060000_rank5.pt: 26%|██▌ | 636MB / 2.49GB  ...nts/optim_060000_rank6.pt: 26%|██▌ | 639MB / 2.49GB  ...nts/optim_060000_rank7.pt: 26%|██▌ | 645MB / 2.49GB  ...nts/optim_060000_rank3.pt: 26%|██▌ | 644MB / 2.49GB  Processing Files (0 / 8) : 21%|██ | 4.71GB / 22.2GB, 455MB/s New Data Upload : 57%|█████▋ | 1.35GB / 2.35GB, 125MB/s  ...eckpoints/model_060000.pt: 9%|▉ | 207MB / 2.35GB  ...nts/optim_060000_rank0.pt: 28%|██▊ | 688MB / 2.49GB  ...nts/optim_060000_rank1.pt: 26%|██▋ | 656MB / 2.49GB  ...nts/optim_060000_rank4.pt: 26%|██▌ | 652MB / 2.49GB  ...nts/optim_060000_rank5.pt: 26%|██▋ | 656MB / 2.49GB  ...nts/optim_060000_rank6.pt: 27%|██▋ | 660MB / 2.49GB  ...nts/optim_060000_rank7.pt: 27%|██▋ | 664MB / 2.49GB  ...nts/optim_060000_rank3.pt: 27%|██▋ | 664MB / 2.49GB  Processing Files (0 / 8) : 22%|██▏ | 4.85GB / 22.2GB, 468MB/s New Data Upload : 55%|█████▌ | 1.37GB / 2.48GB, 128MB/s  ...eckpoints/model_060000.pt: 9%|▉ | 215MB / 2.35GB  ...nts/optim_060000_rank0.pt: 29%|██▊ | 711MB / 2.49GB  ...nts/optim_060000_rank1.pt: 27%|██▋ | 681MB / 2.49GB  ...nts/optim_060000_rank4.pt: 27%|██▋ | 676MB / 2.49GB  ...nts/optim_060000_rank5.pt: 27%|██▋ | 681MB / 2.49GB  ...nts/optim_060000_rank6.pt: 28%|██▊ | 685MB / 2.49GB  ...nts/optim_060000_rank7.pt: 28%|██▊ | 689MB / 2.49GB  ...nts/optim_060000_rank3.pt: 28%|██▊ | 689MB / 2.49GB  Processing Files (0 / 8) : 23%|██▎ | 5.03GB / 22.2GB, 486MB/s New Data Upload : 57%|█████▋ | 1.41GB / 2.48GB, 131MB/s  ...eckpoints/model_060000.pt: 9%|▉ | 222MB / 2.35GB  ...nts/optim_060000_rank0.pt: 29%|██▉ | 732MB / 2.49GB  ...nts/optim_060000_rank1.pt: 28%|██▊ | 703MB / 2.49GB  ...nts/optim_060000_rank4.pt: 28%|██▊ | 699MB / 2.49GB  ...nts/optim_060000_rank5.pt: 28%|██▊ | 705MB / 2.49GB  ...nts/optim_060000_rank6.pt: 28%|██▊ | 708MB / 2.49GB  ...nts/optim_060000_rank7.pt: 29%|██▊ | 712MB / 2.49GB  ...nts/optim_060000_rank3.pt: 29%|██▊ | 711MB / 2.49GB  Processing Files (0 / 8) : 23%|██▎ | 5.19GB / 22.2GB, 502MB/s New Data Upload : 58%|█████▊ | 1.44GB / 2.48GB, 135MB/s  ...eckpoints/model_060000.pt: 10%|▉ | 231MB / 2.35GB  ...nts/optim_060000_rank0.pt: 30%|███ | 757MB / 2.49GB  ...nts/optim_060000_rank1.pt: 29%|██▉ | 728MB / 2.49GB  ...nts/optim_060000_rank4.pt: 29%|██▉ | 722MB / 2.49GB  ...nts/optim_060000_rank5.pt: 29%|██▉ | 731MB / 2.49GB  ...nts/optim_060000_rank6.pt: 30%|██▉ | 735MB / 2.49GB  ...nts/optim_060000_rank7.pt: 30%|██▉ | 738MB / 2.49GB  ...nts/optim_060000_rank3.pt: 30%|██▉ | 737MB / 2.49GB  Processing Files (0 / 8) : 24%|██▍ | 5.38GB / 22.2GB, 520MB/s New Data Upload : 60%|█████▉ | 1.48GB / 2.48GB, 138MB/s  ...eckpoints/model_060000.pt: 10%|█ | 238MB / 2.35GB  ...nts/optim_060000_rank0.pt: 31%|███▏ | 780MB / 2.49GB  ...nts/optim_060000_rank1.pt: 30%|███ | 751MB / 2.49GB  ...nts/optim_060000_rank4.pt: 30%|██▉ | 745MB / 2.49GB  ...nts/optim_060000_rank5.pt: 30%|███ | 757MB / 2.49GB  ...nts/optim_060000_rank6.pt: 31%|███ | 760MB / 2.49GB  ...nts/optim_060000_rank7.pt: 31%|███ | 763MB / 2.49GB  ...nts/optim_060000_rank3.pt: 31%|███ | 761MB / 2.49GB  Processing Files (0 / 8) : 25%|██▍ | 5.55GB / 22.2GB, 537MB/s New Data Upload : 61%|██████ | 1.52GB / 2.48GB, 141MB/s  ...eckpoints/model_060000.pt: 10%|█ | 243MB / 2.35GB  ...nts/optim_060000_rank0.pt: 32%|███▏ | 799MB / 2.49GB  ...nts/optim_060000_rank1.pt: 31%|███ | 769MB / 2.49GB  ...nts/optim_060000_rank4.pt: 31%|███ | 763MB / 2.49GB  ...nts/optim_060000_rank5.pt: 31%|███▏ | 777MB / 2.49GB  ...nts/optim_060000_rank6.pt: 31%|███▏ | 779MB / 2.49GB  ...nts/optim_060000_rank7.pt: 31%|███▏ | 782MB / 2.49GB  ...nts/optim_060000_rank3.pt: 32%|███▏ | 783MB / 2.49GB  Processing Files (0 / 8) : 26%|██▌ | 5.70GB / 22.2GB, 549MB/s New Data Upload : 61%|██████ | 1.55GB / 2.55GB, 144MB/s  ...eckpoints/model_060000.pt: 11%|█ | 248MB / 2.35GB  ...nts/optim_060000_rank0.pt: 33%|███▎ | 824MB / 2.49GB  ...nts/optim_060000_rank1.pt: 32%|███▏ | 795MB / 2.49GB  ...nts/optim_060000_rank4.pt: 32%|███▏ | 788MB / 2.49GB  ...nts/optim_060000_rank5.pt: 32%|███▏ | 804MB / 2.49GB  ...nts/optim_060000_rank6.pt: 32%|███▏ | 805MB / 2.49GB  ...nts/optim_060000_rank7.pt: 33%|███▎ | 809MB / 2.49GB  ...nts/optim_060000_rank3.pt: 33%|███▎ | 810MB / 2.49GB  Processing Files (0 / 8) : 26%|██▋ | 5.88GB / 22.2GB, 566MB/s New Data Upload : 61%|██████ | 1.58GB / 2.62GB, 146MB/s  ...eckpoints/model_060000.pt: 11%|█ | 253MB / 2.35GB  ...nts/optim_060000_rank0.pt: 34%|███▍ | 846MB / 2.49GB  ...nts/optim_060000_rank1.pt: 33%|███▎ | 817MB / 2.49GB  ...nts/optim_060000_rank4.pt: 32%|███▏ | 807MB / 2.49GB  ...nts/optim_060000_rank5.pt: 33%|███▎ | 826MB / 2.49GB  ...nts/optim_060000_rank6.pt: 33%|███▎ | 829MB / 2.49GB  ...nts/optim_060000_rank7.pt: 33%|███▎ | 833MB / 2.49GB  ...nts/optim_060000_rank3.pt: 34%|███▎ | 833MB / 2.49GB  Processing Files (0 / 8) : 27%|██▋ | 6.04GB / 22.2GB, 580MB/s New Data Upload : 59%|█████▉ | 1.62GB / 2.75GB, 148MB/s  ...eckpoints/model_060000.pt: 11%|█ | 256MB / 2.35GB  ...nts/optim_060000_rank0.pt: 35%|███▌ | 874MB / 2.49GB  ...nts/optim_060000_rank1.pt: 34%|███▍ | 846MB / 2.49GB  ...nts/optim_060000_rank4.pt: 34%|███▎ | 836MB / 2.49GB  ...nts/optim_060000_rank5.pt: 34%|███▍ | 856MB / 2.49GB  ...nts/optim_060000_rank6.pt: 35%|███▍ | 858MB / 2.49GB  ...nts/optim_060000_rank7.pt: 35%|███▍ | 863MB / 2.49GB  ...nts/optim_060000_rank3.pt: 35%|███▍ | 865MB / 2.49GB  Processing Files (0 / 8) : 28%|██▊ | 6.25GB / 22.2GB, 599MB/s New Data Upload : 59%|█████▉ | 1.66GB / 2.82GB, 151MB/s  ...eckpoints/model_060000.pt: 11%|█ | 261MB / 2.35GB  ...nts/optim_060000_rank0.pt: 36%|███▋ | 903MB / 2.49GB  ...nts/optim_060000_rank1.pt: 35%|███▌ | 879MB / 2.49GB  ...nts/optim_060000_rank4.pt: 35%|███▍ | 864MB / 2.49GB  ...nts/optim_060000_rank5.pt: 36%|███▌ | 885MB / 2.49GB  ...nts/optim_060000_rank6.pt: 36%|███▌ | 890MB / 2.49GB  ...nts/optim_060000_rank7.pt: 36%|███▌ | 895MB / 2.49GB  ...nts/optim_060000_rank3.pt: 36%|███▌ | 898MB / 2.49GB  Processing Files (0 / 8) : 29%|██▉ | 6.47GB / 22.2GB, 618MB/s New Data Upload : 60%|██████ | 1.70GB / 2.82GB, 154MB/s  ...eckpoints/model_060000.pt: 12%|█▏ | 270MB / 2.35GB  ...nts/optim_060000_rank0.pt: 37%|███▋ | 931MB / 2.49GB  ...nts/optim_060000_rank1.pt: 37%|███▋ | 910MB / 2.49GB  ...nts/optim_060000_rank4.pt: 36%|███▌ | 895MB / 2.49GB  ...nts/optim_060000_rank5.pt: 37%|███▋ | 915MB / 2.49GB  ...nts/optim_060000_rank6.pt: 37%|███▋ | 921MB / 2.49GB  ...nts/optim_060000_rank7.pt: 37%|███▋ | 926MB / 2.49GB  ...nts/optim_060000_rank3.pt: 37%|███▋ | 930MB / 2.49GB  Processing Files (0 / 8) : 30%|███ | 6.70GB / 22.2GB, 638MB/s New Data Upload : 59%|█████▉ | 1.75GB / 2.95GB, 157MB/s  ...eckpoints/model_060000.pt: 12%|█▏ | 274MB / 2.35GB  ...nts/optim_060000_rank0.pt: 39%|███▊ | 960MB / 2.49GB  ...nts/optim_060000_rank1.pt: 38%|███▊ | 941MB / 2.49GB  ...nts/optim_060000_rank4.pt: 37%|███▋ | 924MB / 2.49GB  ...nts/optim_060000_rank5.pt: 38%|███▊ | 945MB / 2.49GB  ...nts/optim_060000_rank6.pt: 38%|███▊ | 952MB / 2.49GB  ...nts/optim_060000_rank7.pt: 39%|███▊ | 958MB / 2.49GB  ...nts/optim_060000_rank3.pt: 39%|███▉ | 965MB / 2.49GB  Processing Files (0 / 8) : 31%|███ | 6.92GB / 22.2GB, 657MB/s New Data Upload : 61%|██████ | 1.80GB / 2.95GB, 160MB/s  ...eckpoints/model_060000.pt: 12%|█▏ | 276MB / 2.35GB  ...nts/optim_060000_rank0.pt: 40%|███▉ | 983MB / 2.49GB  ...nts/optim_060000_rank1.pt: 39%|███▉ | 968MB / 2.49GB  ...nts/optim_060000_rank4.pt: 38%|███▊ | 950MB / 2.49GB  ...nts/optim_060000_rank5.pt: 39%|███▉ | 969MB / 2.49GB  ...nts/optim_060000_rank6.pt: 39%|███▉ | 977MB / 2.49GB  ...nts/optim_060000_rank7.pt: 40%|███▉ | 983MB / 2.49GB  ...nts/optim_060000_rank3.pt: 40%|███▉ | 989MB / 2.49GB  Processing Files (0 / 8) : 32%|███▏ | 7.09GB / 22.2GB, 670MB/s New Data Upload : 62%|██████▏ | 1.83GB / 2.95GB, 161MB/s  ...eckpoints/model_060000.pt: 12%|█▏ | 280MB / 2.35GB  ...nts/optim_060000_rank0.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_060000_rank1.pt: 40%|████ | 1.00GB / 2.49GB  ...nts/optim_060000_rank4.pt: 40%|███▉ | 983MB / 2.49GB  ...nts/optim_060000_rank5.pt: 40%|████ | 998MB / 2.49GB  ...nts/optim_060000_rank6.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_060000_rank7.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_060000_rank3.pt: 41%|████ | 1.02GB / 2.49GB  Processing Files (0 / 8) : 33%|███▎ | 7.32GB / 22.2GB, 687MB/s New Data Upload : 64%|██████▎ | 1.88GB / 2.95GB, 162MB/s  ...eckpoints/model_060000.pt: 12%|█▏ | 283MB / 2.35GB  ...nts/optim_060000_rank0.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_060000_rank1.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_060000_rank4.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_060000_rank5.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_060000_rank6.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_060000_rank7.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_060000_rank3.pt: 42%|████▏ | 1.05GB / 2.49GB  Processing Files (0 / 8) : 34%|███▍ | 7.52GB / 22.2GB, 704MB/s New Data Upload : 64%|██████▎ | 1.92GB / 3.02GB, 165MB/s  ...eckpoints/model_060000.pt: 12%|█▏ | 286MB / 2.35GB  ...nts/optim_060000_rank0.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_060000_rank1.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_060000_rank4.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_060000_rank5.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_060000_rank6.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_060000_rank7.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_060000_rank3.pt: 43%|████▎ | 1.08GB / 2.49GB  Processing Files (0 / 8) : 35%|███▍ | 7.72GB / 22.2GB, 721MB/s New Data Upload : 63%|██████▎ | 1.96GB / 3.08GB, 168MB/s  ...eckpoints/model_060000.pt: 12%|█▏ | 290MB / 2.35GB  ...nts/optim_060000_rank0.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_060000_rank1.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_060000_rank4.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_060000_rank5.pt: 44%|████▎ | 1.08GB / 2.49GB  ...nts/optim_060000_rank6.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_060000_rank7.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_060000_rank3.pt: 45%|████▍ | 1.11GB / 2.49GB  Processing Files (0 / 8) : 36%|███▌ | 7.97GB / 22.2GB, 742MB/s New Data Upload : 60%|█████▉ | 2.01GB / 3.35GB, 170MB/s  ...eckpoints/model_060000.pt: 13%|█▎ | 296MB / 2.35GB  ...nts/optim_060000_rank0.pt: 45%|████▌ | 1.12GB / 2.49GB  ...nts/optim_060000_rank1.pt: 45%|████▌ | 1.13GB / 2.49GB  ...nts/optim_060000_rank4.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_060000_rank5.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_060000_rank6.pt: 45%|████▌ | 1.13GB / 2.49GB  ...nts/optim_060000_rank7.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_060000_rank3.pt: 46%|████▌ | 1.14GB / 2.49GB  Processing Files (0 / 8) : 37%|███▋ | 8.15GB / 22.2GB, 758MB/s New Data Upload : 59%|█████▉ | 2.05GB / 3.49GB, 173MB/s  ...eckpoints/model_060000.pt: 13%|█▎ | 299MB / 2.35GB  ...nts/optim_060000_rank0.pt: 46%|████▌ | 1.15GB / 2.49GB  ...nts/optim_060000_rank1.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_060000_rank4.pt: 45%|████▌ | 1.12GB / 2.49GB  ...nts/optim_060000_rank5.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_060000_rank6.pt: 46%|████▋ | 1.16GB / 2.49GB  ...nts/optim_060000_rank7.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_060000_rank3.pt: 47%|████▋ | 1.17GB / 2.49GB  Processing Files (0 / 8) : 38%|███▊ | 8.35GB / 22.2GB, 775MB/s New Data Upload : 60%|██████ | 2.09GB / 3.49GB, 176MB/s  ...eckpoints/model_060000.pt: 13%|█▎ | 305MB / 2.35GB  ...nts/optim_060000_rank0.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_060000_rank1.pt: 48%|████▊ | 1.18GB / 2.49GB  ...nts/optim_060000_rank4.pt: 46%|████▌ | 1.15GB / 2.49GB  ...nts/optim_060000_rank5.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_060000_rank6.pt: 48%|████▊ | 1.18GB / 2.49GB  ...nts/optim_060000_rank7.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_060000_rank3.pt: 48%|████▊ | 1.19GB / 2.49GB  Processing Files (0 / 8) : 38%|███▊ | 8.53GB / 22.2GB, 790MB/s New Data Upload : 59%|█████▉ | 2.13GB / 3.62GB, 179MB/s  ...eckpoints/model_060000.pt: 13%|█▎ | 309MB / 2.35GB  ...nts/optim_060000_rank0.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_060000_rank1.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_060000_rank4.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_060000_rank5.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_060000_rank6.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_060000_rank7.pt: 49%|████▊ | 1.21GB / 2.49GB  ...nts/optim_060000_rank3.pt: 48%|████▊ | 1.20GB / 2.49GB  Processing Files (0 / 8) : 39%|███▉ | 8.64GB / 22.2GB, 797MB/s New Data Upload : 60%|█████▉ | 2.16GB / 3.62GB, 179MB/s  ...eckpoints/model_060000.pt: 14%|█▎ | 318MB / 2.35GB  ...nts/optim_060000_rank0.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_060000_rank1.pt: 50%|████▉ | 1.23GB / 2.49GB  ...nts/optim_060000_rank4.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_060000_rank5.pt: 49%|████▊ | 1.21GB / 2.49GB  ...nts/optim_060000_rank6.pt: 49%|████▉ | 1.23GB / 2.49GB  ...nts/optim_060000_rank7.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_060000_rank3.pt: 50%|████▉ | 1.24GB / 2.49GB  Processing Files (0 / 8) : 40%|███▉ | 8.87GB / 22.2GB, 817MB/s New Data Upload : 61%|██████ | 2.21GB / 3.62GB, 184MB/s  ...eckpoints/model_060000.pt: 14%|█▍ | 324MB / 2.35GB  ...nts/optim_060000_rank0.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_060000_rank1.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_060000_rank4.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_060000_rank5.pt: 49%|████▉ | 1.23GB / 2.49GB  ...nts/optim_060000_rank6.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_060000_rank7.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_060000_rank3.pt: 50%|█████ | 1.25GB / 2.49GB  Processing Files (0 / 8) : 41%|████ | 9.02GB / 22.2GB, 824MB/s New Data Upload : 61%|██████ | 2.25GB / 3.69GB, 184MB/s  ...eckpoints/model_060000.pt: 14%|█▍ | 338MB / 2.35GB  ...nts/optim_060000_rank0.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_060000_rank1.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_060000_rank4.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_060000_rank5.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_060000_rank6.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_060000_rank7.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_060000_rank3.pt: 51%|█████ | 1.26GB / 2.49GB  Processing Files (0 / 8) : 41%|████ | 9.13GB / 22.2GB, 831MB/s New Data Upload : 61%|██████ | 2.29GB / 3.76GB, 185MB/s  ...eckpoints/model_060000.pt: 15%|█▍ | 346MB / 2.35GB  ...nts/optim_060000_rank0.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_060000_rank1.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_060000_rank4.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_060000_rank5.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_060000_rank6.pt: 52%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_060000_rank7.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_060000_rank3.pt: 52%|█████▏ | 1.28GB / 2.49GB  Processing Files (0 / 8) : 42%|████▏ | 9.28GB / 22.2GB, 840MB/s New Data Upload : 62%|██████▏ | 2.32GB / 3.76GB, 187MB/s  ...eckpoints/model_060000.pt: 15%|█▍ | 352MB / 2.35GB  ...nts/optim_060000_rank0.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_060000_rank1.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_060000_rank4.pt: 52%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_060000_rank5.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_060000_rank6.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_060000_rank7.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_060000_rank3.pt: 52%|█████▏ | 1.30GB / 2.49GB  Processing Files (0 / 8) : 42%|████▏ | 9.41GB / 22.2GB, 844MB/s New Data Upload : 62%|██████▏ | 2.37GB / 3.82GB, 188MB/s  ...eckpoints/model_060000.pt: 15%|█▌ | 361MB / 2.35GB  ...nts/optim_060000_rank0.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_060000_rank1.pt: 54%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_060000_rank4.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_060000_rank5.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_060000_rank6.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_060000_rank7.pt: 53%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_060000_rank3.pt: 53%|█████▎ | 1.32GB / 2.49GB  Processing Files (0 / 8) : 43%|████▎ | 9.56GB / 22.2GB, 850MB/s New Data Upload : 63%|██████▎ | 2.41GB / 3.82GB, 189MB/s  ...eckpoints/model_060000.pt: 16%|█▌ | 373MB / 2.35GB  ...nts/optim_060000_rank0.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_060000_rank1.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_060000_rank4.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_060000_rank5.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_060000_rank6.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_060000_rank7.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_060000_rank3.pt: 54%|█████▎ | 1.33GB / 2.49GB  Processing Files (0 / 8) : 44%|████▎ | 9.71GB / 22.2GB, 854MB/s New Data Upload : 64%|██████▍ | 2.46GB / 3.82GB, 191MB/s  ...eckpoints/model_060000.pt: 16%|█▋ | 385MB / 2.35GB  ...nts/optim_060000_rank0.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_060000_rank1.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_060000_rank4.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_060000_rank5.pt: 53%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_060000_rank6.pt: 55%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_060000_rank7.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_060000_rank3.pt: 54%|█████▍ | 1.35GB / 2.49GB  Processing Files (0 / 8) : 44%|████▍ | 9.82GB / 22.2GB, 851MB/s New Data Upload : 65%|██████▌ | 2.50GB / 3.82GB, 191MB/s  ...eckpoints/model_060000.pt: 17%|█▋ | 398MB / 2.35GB  ...nts/optim_060000_rank0.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_060000_rank1.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_060000_rank4.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_060000_rank5.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_060000_rank6.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_060000_rank7.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_060000_rank3.pt: 55%|█████▍ | 1.37GB / 2.49GB  Processing Files (0 / 8) : 45%|████▍ | 9.99GB / 22.2GB, 857MB/s New Data Upload : 66%|██████▌ | 2.55GB / 3.89GB, 193MB/s  ...eckpoints/model_060000.pt: 17%|█▋ | 409MB / 2.35GB  ...nts/optim_060000_rank0.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_060000_rank1.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...nts/optim_060000_rank4.pt: 56%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_060000_rank5.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_060000_rank6.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_060000_rank7.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...nts/optim_060000_rank3.pt: 56%|█████▌ | 1.39GB / 2.49GB  Processing Files (0 / 8) : 46%|████▌ | 10.1GB / 22.2GB, 860MB/s New Data Upload : 66%|██████▌ | 2.60GB / 3.96GB, 195MB/s  ...eckpoints/model_060000.pt: 18%|█▊ | 412MB / 2.35GB  ...nts/optim_060000_rank0.pt: 56%|█████▌ | 1.40GB / 2.49GB  ...nts/optim_060000_rank1.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_060000_rank4.pt: 57%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_060000_rank5.pt: 56%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_060000_rank6.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_060000_rank7.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_060000_rank3.pt: 56%|█████▋ | 1.40GB / 2.49GB  Processing Files (0 / 8) : 46%|████▌ | 10.3GB / 22.2GB, 862MB/s New Data Upload : 62%|██████▏ | 2.64GB / 4.22GB, 196MB/s  ...eckpoints/model_060000.pt: 18%|█▊ | 422MB / 2.35GB  ...nts/optim_060000_rank0.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_060000_rank1.pt: 59%|█████▉ | 1.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_060000_rank5.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...nts/optim_060000_rank6.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_060000_rank7.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_060000_rank3.pt: 58%|█████▊ | 1.43GB / 2.49GB  Processing Files (0 / 8) : 47%|████▋ | 10.5GB / 22.2GB, 870MB/s New Data Upload : 63%|██████▎ | 2.69GB / 4.29GB, 198MB/s  ...eckpoints/model_060000.pt: 18%|█▊ | 429MB / 2.35GB  ...nts/optim_060000_rank0.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_060000_rank1.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 59%|█████▉ | 1.46GB / 2.49GB  ...nts/optim_060000_rank5.pt: 58%|█████▊ | 1.43GB / 2.49GB  ...nts/optim_060000_rank6.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_060000_rank7.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 59%|█████▊ | 1.46GB / 2.49GB  Processing Files (0 / 8) : 48%|████▊ | 10.7GB / 22.2GB, 871MB/s New Data Upload : 64%|██████▍ | 2.74GB / 4.29GB, 199MB/s  ...eckpoints/model_060000.pt: 19%|█▊ | 438MB / 2.35GB  ...nts/optim_060000_rank0.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_060000_rank4.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_060000_rank6.pt: 60%|██████ | 1.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_060000_rank3.pt: 59%|█████▉ | 1.48GB / 2.49GB  Processing Files (0 / 8) : 49%|████▊ | 10.8GB / 22.2GB, 873MB/s New Data Upload : 63%|██████▎ | 2.79GB / 4.43GB, 201MB/s  ...eckpoints/model_060000.pt: 19%|█▉ | 448MB / 2.35GB  ...nts/optim_060000_rank0.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_060000_rank4.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_060000_rank5.pt: 59%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 61%|██████ | 1.52GB / 2.49GB  ...nts/optim_060000_rank7.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_060000_rank3.pt: 60%|██████ | 1.50GB / 2.49GB  Processing Files (0 / 8) : 49%|████▉ | 11.0GB / 22.2GB, 875MB/s New Data Upload : 63%|██████▎ | 2.84GB / 4.49GB, 202MB/s  ...eckpoints/model_060000.pt: 20%|█▉ | 459MB / 2.35GB  ...nts/optim_060000_rank0.pt: 61%|██████ | 1.52GB / 2.49GB  ...nts/optim_060000_rank1.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_060000_rank4.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_060000_rank5.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_060000_rank6.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_060000_rank7.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_060000_rank3.pt: 62%|██████▏ | 1.53GB / 2.49GB  Processing Files (0 / 8) : 50%|█████ | 11.2GB / 22.2GB, 878MB/s New Data Upload : 64%|██████▍ | 2.89GB / 4.49GB, 204MB/s  ...eckpoints/model_060000.pt: 20%|██ | 470MB / 2.35GB  ...nts/optim_060000_rank0.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_060000_rank1.pt: 64%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_060000_rank4.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_060000_rank5.pt: 61%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_060000_rank6.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_060000_rank7.pt: 64%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_060000_rank3.pt: 62%|██████▏ | 1.55GB / 2.49GB  Processing Files (0 / 8) : 51%|█████ | 11.4GB / 22.2GB, 883MB/s New Data Upload : 64%|██████▎ | 2.94GB / 4.63GB, 207MB/s  ...eckpoints/model_060000.pt: 20%|██ | 478MB / 2.35GB  ...nts/optim_060000_rank0.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_060000_rank1.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_060000_rank4.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_060000_rank5.pt: 63%|██████▎ | 1.55GB / 2.49GB  ...nts/optim_060000_rank6.pt: 64%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_060000_rank7.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_060000_rank3.pt: 63%|██████▎ | 1.58GB / 2.49GB  Processing Files (0 / 8) : 52%|█████▏ | 11.6GB / 22.2GB, 886MB/s New Data Upload : 65%|██████▍ | 2.99GB / 4.63GB, 209MB/s  ...eckpoints/model_060000.pt: 21%|██ | 486MB / 2.35GB  ...nts/optim_060000_rank0.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_060000_rank1.pt: 65%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_060000_rank4.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_060000_rank5.pt: 63%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_060000_rank6.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_060000_rank7.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_060000_rank3.pt: 64%|██████▍ | 1.60GB / 2.49GB  Processing Files (0 / 8) : 53%|█████▎ | 11.7GB / 22.2GB, 887MB/s New Data Upload : 66%|██████▌ | 3.04GB / 4.63GB, 211MB/s  ...eckpoints/model_060000.pt: 21%|██ | 496MB / 2.35GB  ...nts/optim_060000_rank0.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_060000_rank1.pt: 67%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_060000_rank4.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_060000_rank5.pt: 65%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_060000_rank6.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_060000_rank7.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_060000_rank3.pt: 65%|██████▌ | 1.62GB / 2.49GB  Processing Files (0 / 8) : 54%|█████▎ | 11.9GB / 22.2GB, 892MB/s New Data Upload : 66%|██████▌ | 3.09GB / 4.69GB, 213MB/s  ...eckpoints/model_060000.pt: 21%|██▏ | 502MB / 2.35GB  ...nts/optim_060000_rank0.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_060000_rank1.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_060000_rank4.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_060000_rank5.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_060000_rank6.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_060000_rank7.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_060000_rank3.pt: 66%|██████▋ | 1.65GB / 2.49GB  Processing Files (0 / 8) : 54%|█████▍ | 12.1GB / 22.2GB, 891MB/s New Data Upload : 66%|██████▌ | 3.15GB / 4.76GB, 214MB/s  ...eckpoints/model_060000.pt: 22%|██▏ | 508MB / 2.35GB  ...nts/optim_060000_rank0.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_060000_rank1.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_060000_rank4.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_060000_rank5.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_060000_rank6.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_060000_rank7.pt: 69%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_060000_rank3.pt: 67%|██████▋ | 1.67GB / 2.49GB  Processing Files (0 / 8) : 55%|█████▌ | 12.2GB / 22.2GB, 887MB/s New Data Upload : 67%|██████▋ | 3.20GB / 4.76GB, 214MB/s  ...nts/optim_060000_rank2.pt: 4%|▍ | 95.9MB / 2.49GB  ...eckpoints/model_060000.pt: 22%|██▏ | 512MB / 2.35GB  ...nts/optim_060000_rank0.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_060000_rank1.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_060000_rank4.pt: 69%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_060000_rank5.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_060000_rank6.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_060000_rank7.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_060000_rank3.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_060000_rank2.pt: 4%|▍ | 95.9MB / 2.49GB  Processing Files (0 / 9) : 56%|█████▌ | 12.5GB / 22.2GB, 894MB/s New Data Upload : 68%|██████▊ | 3.24GB / 4.76GB, 214MB/s  ...eckpoints/model_060000.pt: 22%|██▏ | 521MB / 2.35GB  ...nts/optim_060000_rank0.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_060000_rank1.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_060000_rank4.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_060000_rank5.pt: 69%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_060000_rank6.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_060000_rank7.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_060000_rank3.pt: 69%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_060000_rank2.pt: 9%|▊ | 216MB / 2.49GB  Processing Files (0 / 9) : 58%|█████▊ | 12.8GB / 22.2GB, 912MB/s New Data Upload : 69%|██████▊ | 3.30GB / 4.82GB, 217MB/s  ...eckpoints/model_060000.pt: 22%|██▏ | 526MB / 2.35GB  ...nts/optim_060000_rank0.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_060000_rank1.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_060000_rank4.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_060000_rank5.pt: 69%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_060000_rank6.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_060000_rank7.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_060000_rank3.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_060000_rank2.pt: 14%|█▍ | 351MB / 2.49GB  Processing Files (0 / 9) : 59%|█████▉ | 13.1GB / 22.2GB, 927MB/s New Data Upload : 70%|██████▉ | 3.35GB / 4.82GB, 219MB/s  ...eckpoints/model_060000.pt: 23%|██▎ | 529MB / 2.35GB  ...nts/optim_060000_rank0.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_060000_rank1.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_060000_rank4.pt: 71%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_060000_rank5.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_060000_rank6.pt: 71%|███████ | 1.75GB / 2.49GB  ...nts/optim_060000_rank7.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_060000_rank3.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_060000_rank2.pt: 20%|█▉ | 487MB / 2.49GB  Processing Files (0 / 9) : 60%|██████ | 13.4GB / 22.2GB, 933MB/s New Data Upload : 69%|██████▊ | 3.40GB / 4.95GB, 218MB/s  ...eckpoints/model_060000.pt: 23%|██▎ | 538MB / 2.35GB  ...nts/optim_060000_rank0.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_060000_rank1.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_060000_rank4.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_060000_rank5.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_060000_rank6.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_060000_rank7.pt: 73%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_060000_rank3.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_060000_rank2.pt: 24%|██▍ | 604MB / 2.49GB  Processing Files (0 / 9) : 61%|██████▏ | 13.7GB / 22.2GB, 936MB/s New Data Upload : 70%|██████▉ | 3.44GB / 4.95GB, 218MB/s  ...eckpoints/model_060000.pt: 23%|██▎ | 542MB / 2.35GB  ...nts/optim_060000_rank0.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_060000_rank1.pt: 74%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_060000_rank4.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_060000_rank5.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_060000_rank6.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_060000_rank7.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_060000_rank3.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_060000_rank2.pt: 30%|███ | 747MB / 2.49GB  Processing Files (0 / 9) : 63%|██████▎ | 13.9GB / 22.2GB, 945MB/s New Data Upload : 70%|███████ | 3.49GB / 4.95GB, 219MB/s  ...eckpoints/model_060000.pt: 23%|██▎ | 550MB / 2.35GB  ...nts/optim_060000_rank0.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_060000_rank1.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_060000_rank4.pt: 74%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_060000_rank5.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_060000_rank6.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_060000_rank7.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_060000_rank3.pt: 73%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_060000_rank2.pt: 33%|███▎ | 827MB / 2.49GB  Processing Files (0 / 9) : 64%|██████▎ | 14.1GB / 22.2GB, 943MB/s New Data Upload : 71%|███████ | 3.52GB / 4.95GB, 217MB/s  ...eckpoints/model_060000.pt: 24%|██▎ | 556MB / 2.35GB  ...nts/optim_060000_rank0.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_060000_rank1.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_060000_rank4.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_060000_rank5.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_060000_rank6.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_060000_rank7.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_060000_rank3.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_060000_rank2.pt: 40%|████ | 997MB / 2.49GB  Processing Files (0 / 9) : 65%|██████▌ | 14.5GB / 22.2GB, 957MB/s New Data Upload : 72%|███████▏ | 3.57GB / 4.95GB, 218MB/s  ...eckpoints/model_060000.pt: 24%|██▍ | 567MB / 2.35GB  ...nts/optim_060000_rank0.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_060000_rank1.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_060000_rank4.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_060000_rank5.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_060000_rank6.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_060000_rank7.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_060000_rank3.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_060000_rank2.pt: 45%|████▍ | 1.11GB / 2.49GB  Processing Files (0 / 9) : 66%|██████▌ | 14.7GB / 22.2GB, 968MB/s New Data Upload : 73%|███████▎ | 3.62GB / 4.95GB, 220MB/s  ...eckpoints/model_060000.pt: 25%|██▍ | 577MB / 2.35GB  ...nts/optim_060000_rank0.pt: 75%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_060000_rank1.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_060000_rank4.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_060000_rank5.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_060000_rank6.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_060000_rank7.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_060000_rank3.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_060000_rank2.pt: 47%|████▋ | 1.17GB / 2.49GB  Processing Files (0 / 9) : 67%|██████▋ | 14.9GB / 22.2GB, 970MB/s New Data Upload : 74%|███████▍ | 3.66GB / 4.95GB, 221MB/s  ...eckpoints/model_060000.pt: 25%|██▍ | 586MB / 2.35GB  ...nts/optim_060000_rank0.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_060000_rank1.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_060000_rank4.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_060000_rank5.pt: 75%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_060000_rank6.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_060000_rank7.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_060000_rank3.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_060000_rank2.pt: 48%|████▊ | 1.20GB / 2.49GB  Processing Files (0 / 9) : 68%|██████▊ | 15.1GB / 22.2GB, 972MB/s New Data Upload : 75%|███████▍ | 3.71GB / 4.95GB, 222MB/s  ...eckpoints/model_060000.pt: 25%|██▌ | 596MB / 2.35GB  ...nts/optim_060000_rank0.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_060000_rank1.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_060000_rank4.pt: 77%|███████▋ | 1.93GB / 2.49GB  ...nts/optim_060000_rank5.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_060000_rank6.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_060000_rank7.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_060000_rank3.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_060000_rank2.pt: 53%|█████▎ | 1.31GB / 2.49GB  Processing Files (0 / 9) : 69%|██████▉ | 15.3GB / 22.2GB, 977MB/s New Data Upload : 75%|███████▍ | 3.75GB / 5.01GB, 223MB/s  ...eckpoints/model_060000.pt: 26%|██▌ | 600MB / 2.35GB  ...nts/optim_060000_rank0.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_060000_rank1.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_060000_rank4.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_060000_rank5.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_060000_rank6.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_060000_rank7.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_060000_rank3.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_060000_rank2.pt: 54%|█████▍ | 1.34GB / 2.49GB  Processing Files (0 / 9) : 70%|██████▉ | 15.5GB / 22.2GB, 978MB/s New Data Upload : 76%|███████▌ | 3.79GB / 5.01GB, 223MB/s  ...eckpoints/model_060000.pt: 26%|██▌ | 607MB / 2.35GB  ...nts/optim_060000_rank0.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_060000_rank1.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_060000_rank4.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_060000_rank5.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_060000_rank6.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_060000_rank7.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_060000_rank3.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_060000_rank2.pt: 58%|█████▊ | 1.45GB / 2.49GB  Processing Files (0 / 9) : 71%|███████ | 15.8GB / 22.2GB, 990MB/s New Data Upload : 77%|███████▋ | 3.84GB / 5.01GB, 224MB/s  ...eckpoints/model_060000.pt: 27%|██▋ | 624MB / 2.35GB  ...nts/optim_060000_rank0.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_060000_rank1.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_060000_rank4.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_060000_rank5.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_060000_rank6.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_060000_rank7.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_060000_rank3.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_060000_rank2.pt: 68%|██████▊ | 1.69GB / 2.49GB  Processing Files (0 / 9) : 73%|███████▎ | 16.2GB / 22.2GB, 1.01GB/s New Data Upload : 77%|███████▋ | 3.89GB / 5.08GB, 226MB/s  ...eckpoints/model_060000.pt: 27%|██▋ | 632MB / 2.35GB  ...nts/optim_060000_rank0.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_060000_rank1.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_060000_rank4.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_060000_rank5.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_060000_rank6.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_060000_rank7.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_060000_rank3.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_060000_rank2.pt: 71%|███████ | 1.75GB / 2.49GB  Processing Files (0 / 9) : 74%|███████▍ | 16.4GB / 22.2GB, 1.02GB/s New Data Upload : 77%|███████▋ | 3.93GB / 5.08GB, 227MB/s  ...eckpoints/model_060000.pt: 27%|██▋ | 645MB / 2.35GB  ...nts/optim_060000_rank0.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_060000_rank1.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_060000_rank4.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_060000_rank5.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_060000_rank6.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_060000_rank7.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_060000_rank3.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_060000_rank2.pt: 73%|███████▎ | 1.82GB / 2.49GB  Processing Files (0 / 9) : 75%|███████▍ | 16.6GB / 22.2GB, 1.02GB/s New Data Upload : 78%|███████▊ | 3.98GB / 5.08GB, 227MB/s  ...eckpoints/model_060000.pt: 28%|██▊ | 653MB / 2.35GB  ...nts/optim_060000_rank0.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_060000_rank1.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_060000_rank4.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_060000_rank5.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_060000_rank6.pt: 81%|████████ | 2.00GB / 2.49GB  ...nts/optim_060000_rank7.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_060000_rank3.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_060000_rank2.pt: 76%|███████▌ | 1.89GB / 2.49GB  Processing Files (0 / 9) : 76%|███████▌ | 16.8GB / 22.2GB, 1.01GB/s New Data Upload : 78%|███████▊ | 4.02GB / 5.15GB, 227MB/s  ...eckpoints/model_060000.pt: 28%|██▊ | 663MB / 2.35GB  ...nts/optim_060000_rank0.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_060000_rank1.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_060000_rank4.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_060000_rank5.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_060000_rank6.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_060000_rank7.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_060000_rank3.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_060000_rank2.pt: 79%|███████▊ | 1.96GB / 2.49GB  Processing Files (0 / 9) : 77%|███████▋ | 17.0GB / 22.2GB, 1.01GB/s New Data Upload : 78%|███████▊ | 4.06GB / 5.22GB, 226MB/s  ...eckpoints/model_060000.pt: 29%|██▊ | 675MB / 2.35GB  ...nts/optim_060000_rank0.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_060000_rank1.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_060000_rank4.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_060000_rank5.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_060000_rank6.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_060000_rank7.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_060000_rank3.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_060000_rank2.pt: 83%|████████▎ | 2.06GB / 2.49GB  Processing Files (0 / 9) : 78%|███████▊ | 17.2GB / 22.2GB, 1.01GB/s New Data Upload : 77%|███████▋ | 4.09GB / 5.28GB, 225MB/s  ...eckpoints/model_060000.pt: 29%|██▉ | 690MB / 2.35GB  ...nts/optim_060000_rank0.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_060000_rank1.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_060000_rank4.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_060000_rank5.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_060000_rank6.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_060000_rank7.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_060000_rank3.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_060000_rank2.pt: 84%|████████▎ | 2.08GB / 2.49GB  Processing Files (0 / 9) : 78%|███████▊ | 17.4GB / 22.2GB, 1.01GB/s New Data Upload : 78%|███████▊ | 4.14GB / 5.28GB, 226MB/s  ...eckpoints/model_060000.pt: 30%|███ | 712MB / 2.35GB  ...nts/optim_060000_rank0.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_060000_rank1.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_060000_rank4.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_060000_rank5.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_060000_rank6.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_060000_rank7.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_060000_rank3.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_060000_rank2.pt: 84%|████████▍ | 2.09GB / 2.49GB  Processing Files (0 / 9) : 79%|███████▉ | 17.5GB / 22.2GB, 1.00GB/s New Data Upload : 79%|███████▉ | 4.19GB / 5.28GB, 226MB/s  ...eckpoints/model_060000.pt: 31%|███ | 726MB / 2.35GB  ...nts/optim_060000_rank0.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_060000_rank1.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_060000_rank4.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_060000_rank5.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_060000_rank6.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_060000_rank7.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_060000_rank3.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_060000_rank2.pt: 85%|████████▌ | 2.11GB / 2.49GB  Processing Files (0 / 9) : 80%|███████▉ | 17.7GB / 22.2GB, 997MB/s New Data Upload : 80%|████████ | 4.23GB / 5.28GB, 227MB/s  ...eckpoints/model_060000.pt: 32%|███▏ | 741MB / 2.35GB  ...nts/optim_060000_rank0.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_060000_rank1.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_060000_rank4.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_060000_rank5.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_060000_rank6.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_060000_rank7.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_060000_rank3.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_060000_rank2.pt: 85%|████████▌ | 2.12GB / 2.49GB  Processing Files (0 / 9) : 80%|███████▉ | 17.8GB / 22.2GB, 985MB/s New Data Upload : 80%|███████▉ | 4.26GB / 5.35GB, 226MB/s  ...eckpoints/model_060000.pt: 32%|███▏ | 759MB / 2.35GB  ...nts/optim_060000_rank0.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_060000_rank1.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_060000_rank4.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_060000_rank5.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_060000_rank6.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_060000_rank7.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_060000_rank3.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_060000_rank2.pt: 86%|████████▌ | 2.14GB / 2.49GB  Processing Files (0 / 9) : 80%|████████ | 17.9GB / 22.2GB, 973MB/s New Data Upload : 80%|████████ | 4.30GB / 5.35GB, 225MB/s  ...eckpoints/model_060000.pt: 33%|███▎ | 778MB / 2.35GB  ...nts/optim_060000_rank0.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_060000_rank1.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_060000_rank4.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_060000_rank5.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_060000_rank6.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_060000_rank7.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_060000_rank3.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_060000_rank2.pt: 86%|████████▋ | 2.14GB / 2.49GB  Processing Files (0 / 9) : 81%|████████ | 18.0GB / 22.2GB, 964MB/s New Data Upload : 80%|████████ | 4.34GB / 5.42GB, 224MB/s  ...eckpoints/model_060000.pt: 34%|███▍ | 794MB / 2.35GB  ...nts/optim_060000_rank0.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_060000_rank1.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_060000_rank4.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_060000_rank5.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_060000_rank6.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_060000_rank7.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_060000_rank3.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_060000_rank2.pt: 87%|████████▋ | 2.15GB / 2.49GB  Processing Files (0 / 9) : 81%|████████▏ | 18.1GB / 22.2GB, 953MB/s New Data Upload : 81%|████████ | 4.37GB / 5.42GB, 223MB/s  ...eckpoints/model_060000.pt: 35%|███▍ | 814MB / 2.35GB  ...nts/optim_060000_rank0.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_060000_rank1.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_060000_rank4.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_060000_rank5.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_060000_rank6.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_060000_rank7.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_060000_rank3.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_060000_rank2.pt: 87%|████████▋ | 2.16GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.2GB / 22.2GB, 943MB/s New Data Upload : 81%|████████▏ | 4.41GB / 5.42GB, 223MB/s  ...eckpoints/model_060000.pt: 36%|███▌ | 841MB / 2.35GB  ...nts/optim_060000_rank0.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_060000_rank1.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_060000_rank4.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_060000_rank5.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_060000_rank6.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_060000_rank7.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_060000_rank3.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_060000_rank2.pt: 87%|████████▋ | 2.17GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.3GB / 22.2GB, 944MB/s New Data Upload : 81%|████████ | 4.45GB / 5.48GB, 225MB/s  ...eckpoints/model_060000.pt: 37%|███▋ | 859MB / 2.35GB  ...nts/optim_060000_rank0.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_060000_rank1.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_060000_rank4.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_060000_rank5.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_060000_rank6.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_060000_rank7.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_060000_rank3.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_060000_rank2.pt: 88%|████████▊ | 2.18GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.4GB / 22.2GB, 928MB/s New Data Upload : 82%|████████▏ | 4.49GB / 5.48GB, 222MB/s  ...eckpoints/model_060000.pt: 37%|███▋ | 877MB / 2.35GB  ...nts/optim_060000_rank0.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_060000_rank1.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_060000_rank4.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_060000_rank5.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_060000_rank6.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_060000_rank7.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_060000_rank3.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_060000_rank2.pt: 88%|████████▊ | 2.19GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.4GB / 22.2GB, 921MB/s New Data Upload : 81%|████████▏ | 4.52GB / 5.55GB, 222MB/s  ...eckpoints/model_060000.pt: 38%|███▊ | 900MB / 2.35GB  ...nts/optim_060000_rank0.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_060000_rank1.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_060000_rank4.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_060000_rank5.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_060000_rank6.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_060000_rank7.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_060000_rank3.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_060000_rank2.pt: 88%|████████▊ | 2.20GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.5GB / 22.2GB, 922MB/s New Data Upload : 82%|████████▏ | 4.56GB / 5.55GB, 223MB/s  ...eckpoints/model_060000.pt: 39%|███▉ | 921MB / 2.35GB  ...nts/optim_060000_rank0.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_060000_rank1.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_060000_rank4.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_060000_rank5.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_060000_rank6.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_060000_rank7.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_060000_rank3.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_060000_rank2.pt: 89%|████████▊ | 2.21GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▎ | 18.6GB / 22.2GB, 914MB/s New Data Upload : 82%|████████▏ | 4.59GB / 5.62GB, 222MB/s  ...eckpoints/model_060000.pt: 40%|███▉ | 937MB / 2.35GB  ...nts/optim_060000_rank0.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_060000_rank1.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_060000_rank4.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_060000_rank5.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_060000_rank6.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_060000_rank7.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_060000_rank3.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_060000_rank2.pt: 89%|████████▉ | 2.21GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▍ | 18.7GB / 22.2GB, 909MB/s New Data Upload : 82%|████████▏ | 4.62GB / 5.62GB, 221MB/s  ...eckpoints/model_060000.pt: 41%|████ | 963MB / 2.35GB  ...nts/optim_060000_rank0.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_060000_rank1.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_060000_rank4.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_060000_rank5.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_060000_rank6.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_060000_rank7.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_060000_rank3.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_060000_rank2.pt: 89%|████████▉ | 2.22GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▍ | 18.8GB / 22.2GB, 901MB/s New Data Upload : 82%|████████▏ | 4.66GB / 5.69GB, 220MB/s  ...eckpoints/model_060000.pt: 42%|████▏ | 980MB / 2.35GB  ...nts/optim_060000_rank0.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_060000_rank1.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_060000_rank4.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_060000_rank5.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_060000_rank6.pt: 89%|████████▊ | 2.21GB / 2.49GB  ...nts/optim_060000_rank7.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_060000_rank3.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_060000_rank2.pt: 90%|████████▉ | 2.23GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▍ | 18.9GB / 22.2GB, 898MB/s New Data Upload : 82%|████████▏ | 4.69GB / 5.75GB, 219MB/s  ...eckpoints/model_060000.pt: 43%|████▎ | 1.00GB / 2.35GB  ...nts/optim_060000_rank0.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_060000_rank1.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_060000_rank4.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_060000_rank5.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_060000_rank6.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_060000_rank7.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_060000_rank3.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_060000_rank2.pt: 90%|█████████ | 2.24GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▌ | 18.9GB / 22.2GB, 895MB/s New Data Upload : 82%|████████▏ | 4.73GB / 5.75GB, 219MB/s  ...eckpoints/model_060000.pt: 43%|████▎ | 1.02GB / 2.35GB  ...nts/optim_060000_rank0.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_060000_rank1.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_060000_rank4.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_060000_rank5.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_060000_rank6.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_060000_rank7.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_060000_rank3.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_060000_rank2.pt: 90%|█████████ | 2.25GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▌ | 19.0GB / 22.2GB, 885MB/s New Data Upload : 82%|████████▏ | 4.76GB / 5.82GB, 217MB/s  ...eckpoints/model_060000.pt: 44%|████▍ | 1.03GB / 2.35GB  ...nts/optim_060000_rank0.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_060000_rank1.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_060000_rank4.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_060000_rank5.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_060000_rank6.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_060000_rank7.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_060000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_060000_rank2.pt: 91%|█████████ | 2.26GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▌ | 19.1GB / 22.2GB, 880MB/s New Data Upload : 82%|████████▏ | 4.79GB / 5.82GB, 215MB/s  ...eckpoints/model_060000.pt: 45%|████▍ | 1.05GB / 2.35GB  ...nts/optim_060000_rank0.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_060000_rank1.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_060000_rank4.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_060000_rank5.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_060000_rank6.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_060000_rank7.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_060000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_060000_rank2.pt: 91%|█████████ | 2.27GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▋ | 19.2GB / 22.2GB, 873MB/s New Data Upload : 83%|████████▎ | 4.82GB / 5.82GB, 214MB/s  ...eckpoints/model_060000.pt: 45%|████▌ | 1.06GB / 2.35GB  ...nts/optim_060000_rank0.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_060000_rank1.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_060000_rank4.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_060000_rank5.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_060000_rank6.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_060000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_060000_rank3.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_060000_rank2.pt: 91%|█████████▏| 2.27GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.3GB / 22.2GB, 860MB/s New Data Upload : 82%|████████▏ | 4.85GB / 5.89GB, 211MB/s  ...eckpoints/model_060000.pt: 46%|████▌ | 1.07GB / 2.35GB  ...nts/optim_060000_rank0.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_060000_rank1.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_060000_rank4.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_060000_rank5.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_060000_rank6.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_060000_rank7.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_060000_rank3.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_060000_rank2.pt: 92%|█████████▏| 2.29GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.4GB / 22.2GB, 851MB/s New Data Upload : 83%|████████▎ | 4.88GB / 5.89GB, 210MB/s  ...eckpoints/model_060000.pt: 47%|████▋ | 1.10GB / 2.35GB  ...nts/optim_060000_rank0.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_060000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_060000_rank4.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_060000_rank5.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_060000_rank6.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_060000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_060000_rank3.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_060000_rank2.pt: 92%|█████████▏| 2.30GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.5GB / 22.2GB, 846MB/s New Data Upload : 83%|████████▎ | 4.92GB / 5.95GB, 209MB/s  ...eckpoints/model_060000.pt: 48%|████▊ | 1.12GB / 2.35GB  ...nts/optim_060000_rank0.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_060000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_060000_rank4.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_060000_rank5.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_060000_rank6.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_060000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_060000_rank3.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_060000_rank2.pt: 93%|█████████▎| 2.30GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.5GB / 22.2GB, 835MB/s New Data Upload : 83%|████████▎ | 4.95GB / 5.95GB, 207MB/s  ...eckpoints/model_060000.pt: 49%|████▊ | 1.14GB / 2.35GB  ...nts/optim_060000_rank0.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_060000_rank1.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_060000_rank4.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_060000_rank5.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_060000_rank6.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_060000_rank7.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_060000_rank3.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_060000_rank2.pt: 93%|█████████▎| 2.31GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.6GB / 22.2GB, 824MB/s New Data Upload : 83%|████████▎ | 4.99GB / 6.02GB, 205MB/s  ...eckpoints/model_060000.pt: 49%|████▉ | 1.16GB / 2.35GB  ...nts/optim_060000_rank0.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_060000_rank1.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_060000_rank4.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_060000_rank5.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_060000_rank6.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_060000_rank7.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_060000_rank3.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_060000_rank2.pt: 93%|█████████▎| 2.32GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▊ | 19.7GB / 22.2GB, 817MB/s New Data Upload : 82%|████████▏ | 5.02GB / 6.09GB, 204MB/s  ...eckpoints/model_060000.pt: 50%|█████ | 1.18GB / 2.35GB  ...nts/optim_060000_rank0.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_060000_rank1.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_060000_rank4.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_060000_rank5.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_060000_rank6.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_060000_rank7.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_060000_rank3.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_060000_rank2.pt: 94%|█████████▎| 2.33GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▉ | 19.8GB / 22.2GB, 807MB/s New Data Upload : 82%|████████▏ | 5.05GB / 6.15GB, 202MB/s  ...eckpoints/model_060000.pt: 51%|█████▏ | 1.21GB / 2.35GB  ...nts/optim_060000_rank0.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_060000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_060000_rank4.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_060000_rank5.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_060000_rank6.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_060000_rank7.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_060000_rank3.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_060000_rank2.pt: 94%|█████████▍| 2.34GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▉ | 19.9GB / 22.2GB, 800MB/s New Data Upload : 83%|████████▎ | 5.09GB / 6.15GB, 201MB/s  ...eckpoints/model_060000.pt: 52%|█████▏ | 1.23GB / 2.35GB  ...nts/optim_060000_rank0.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_060000_rank1.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_060000_rank4.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_060000_rank5.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_060000_rank6.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_060000_rank7.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_060000_rank3.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_060000_rank2.pt: 94%|█████████▍| 2.34GB / 2.49GB  Processing Files (0 / 9) : 90%|████████▉ | 20.0GB / 22.2GB, 789MB/s New Data Upload : 83%|████████▎ | 5.13GB / 6.15GB, 200MB/s  ...eckpoints/model_060000.pt: 54%|█████▎ | 1.26GB / 2.35GB  ...nts/optim_060000_rank0.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_060000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_060000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_060000_rank5.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_060000_rank6.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_060000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_060000_rank3.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_060000_rank2.pt: 95%|█████████▍| 2.35GB / 2.49GB  Processing Files (0 / 9) : 90%|█████████ | 20.0GB / 22.2GB, 779MB/s New Data Upload : 83%|████████▎ | 5.16GB / 6.22GB, 197MB/s  ...eckpoints/model_060000.pt: 54%|█████▍ | 1.27GB / 2.35GB  ...nts/optim_060000_rank0.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_060000_rank1.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_060000_rank4.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_060000_rank5.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_060000_rank6.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_060000_rank7.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_060000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_060000_rank2.pt: 95%|█████████▍| 2.36GB / 2.49GB  Processing Files (0 / 9) : 90%|█████████ | 20.1GB / 22.2GB, 771MB/s New Data Upload : 83%|████████▎ | 5.19GB / 6.29GB, 196MB/s  ...eckpoints/model_060000.pt: 56%|█████▌ | 1.31GB / 2.35GB  ...nts/optim_060000_rank0.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_060000_rank1.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_060000_rank4.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_060000_rank5.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_060000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_060000_rank7.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_060000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_060000_rank2.pt: 95%|█████████▌| 2.36GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.2GB / 22.2GB, 754MB/s New Data Upload : 83%|████████▎ | 5.24GB / 6.29GB, 196MB/s  ...eckpoints/model_060000.pt: 57%|█████▋ | 1.33GB / 2.35GB  ...nts/optim_060000_rank0.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_060000_rank1.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_060000_rank4.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_060000_rank5.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_060000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_060000_rank7.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_060000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_060000_rank2.pt: 95%|█████████▌| 2.37GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.2GB / 22.2GB, 728MB/s New Data Upload : 83%|████████▎ | 5.27GB / 6.36GB, 193MB/s  ...eckpoints/model_060000.pt: 58%|█████▊ | 1.36GB / 2.35GB  ...nts/optim_060000_rank0.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_060000_rank1.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_060000_rank4.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_060000_rank5.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_060000_rank6.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_060000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_060000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_060000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████▏| 20.3GB / 22.2GB, 707MB/s New Data Upload : 83%|████████▎ | 5.31GB / 6.42GB, 192MB/s  ...eckpoints/model_060000.pt: 59%|█████▉ | 1.38GB / 2.35GB  ...nts/optim_060000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_060000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_060000_rank4.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_060000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_060000_rank6.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_060000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_060000_rank3.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_060000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.4GB / 22.2GB, 687MB/s New Data Upload : 82%|████████▏ | 5.35GB / 6.49GB, 191MB/s  ...eckpoints/model_060000.pt: 60%|██████ | 1.42GB / 2.35GB  ...nts/optim_060000_rank0.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_060000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_060000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_060000_rank5.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_060000_rank6.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_060000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_060000_rank3.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_060000_rank2.pt: 96%|█████████▋| 2.39GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.5GB / 22.2GB, 673MB/s New Data Upload : 83%|████████▎ | 5.40GB / 6.50GB, 192MB/s  ...eckpoints/model_060000.pt: 61%|██████ | 1.44GB / 2.35GB  ...nts/optim_060000_rank0.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_060000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_060000_rank4.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_060000_rank5.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_060000_rank6.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_060000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_060000_rank3.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_060000_rank2.pt: 96%|█████████▋| 2.40GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.6GB / 22.2GB, 650MB/s New Data Upload : 83%|████████▎ | 5.42GB / 6.50GB, 190MB/s  ...eckpoints/model_060000.pt: 62%|██████▏ | 1.46GB / 2.35GB  ...nts/optim_060000_rank0.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_060000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_060000_rank4.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_060000_rank5.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_060000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_060000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_060000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_060000_rank2.pt: 97%|█████████▋| 2.40GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.7GB / 22.2GB, 639MB/s New Data Upload : 84%|████████▍ | 5.46GB / 6.50GB, 190MB/s  ...eckpoints/model_060000.pt: 63%|██████▎ | 1.49GB / 2.35GB  ...nts/optim_060000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_060000_rank1.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_060000_rank4.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_060000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_060000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_060000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_060000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_060000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.7GB / 22.2GB, 613MB/s New Data Upload : 84%|████████▍ | 5.49GB / 6.50GB, 188MB/s  ...eckpoints/model_060000.pt: 64%|██████▍ | 1.50GB / 2.35GB  ...nts/optim_060000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_060000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_060000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_060000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_060000_rank6.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_060000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_060000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_060000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.8GB / 22.2GB, 594MB/s New Data Upload : 85%|████████▍ | 5.52GB / 6.50GB, 186MB/s  ...eckpoints/model_060000.pt: 65%|██████▍ | 1.53GB / 2.35GB  ...nts/optim_060000_rank0.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_060000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_060000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_060000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_060000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_060000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_060000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 20.9GB / 22.2GB, 584MB/s New Data Upload : 85%|████████▌ | 5.56GB / 6.50GB, 185MB/s  ...eckpoints/model_060000.pt: 66%|██████▌ | 1.54GB / 2.35GB  ...nts/optim_060000_rank0.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_060000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_060000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_060000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_060000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_060000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 20.9GB / 22.2GB, 571MB/s New Data Upload : 86%|████████▌ | 5.58GB / 6.50GB, 183MB/s  ...eckpoints/model_060000.pt: 67%|██████▋ | 1.57GB / 2.35GB  ...nts/optim_060000_rank0.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_060000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_060000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_060000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_060000_rank7.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 21.0GB / 22.2GB, 553MB/s New Data Upload : 86%|████████▋ | 5.61GB / 6.50GB, 182MB/s  ...eckpoints/model_060000.pt: 68%|██████▊ | 1.59GB / 2.35GB  ...nts/optim_060000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_060000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank4.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_060000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▍| 21.0GB / 22.2GB, 541MB/s New Data Upload : 87%|████████▋ | 5.64GB / 6.50GB, 181MB/s  ...eckpoints/model_060000.pt: 68%|██████▊ | 1.60GB / 2.35GB  ...nts/optim_060000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_060000_rank1.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_060000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▍| 21.1GB / 22.2GB, 520MB/s New Data Upload : 87%|████████▋ | 5.67GB / 6.50GB, 179MB/s  ...eckpoints/model_060000.pt: 69%|██████▉ | 1.63GB / 2.35GB  ...nts/optim_060000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_060000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.2GB / 22.2GB, 483MB/s New Data Upload : 88%|████████▊ | 5.70GB / 6.50GB, 177MB/s  ...eckpoints/model_060000.pt: 70%|███████ | 1.65GB / 2.35GB  ...nts/optim_060000_rank0.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_060000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.2GB / 22.2GB, 470MB/s New Data Upload : 88%|████████▊ | 5.73GB / 6.50GB, 176MB/s  ...eckpoints/model_060000.pt: 71%|███████ | 1.67GB / 2.35GB  ...nts/optim_060000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.2GB / 22.2GB, 453MB/s New Data Upload : 88%|████████▊ | 5.75GB / 6.50GB, 174MB/s  ...eckpoints/model_060000.pt: 72%|███████▏ | 1.69GB / 2.35GB  ...nts/optim_060000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 438MB/s New Data Upload : 89%|████████▉ | 5.77GB / 6.50GB, 172MB/s  ...eckpoints/model_060000.pt: 72%|███████▏ | 1.70GB / 2.35GB  ...nts/optim_060000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 421MB/s New Data Upload : 89%|████████▉ | 5.79GB / 6.50GB, 170MB/s  ...eckpoints/model_060000.pt: 73%|███████▎ | 1.72GB / 2.35GB  ...nts/optim_060000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 402MB/s New Data Upload : 89%|████████▉ | 5.82GB / 6.50GB, 169MB/s  ...eckpoints/model_060000.pt: 74%|███████▍ | 1.74GB / 2.35GB  ...nts/optim_060000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.4GB / 22.2GB, 391MB/s New Data Upload : 90%|████████▉ | 5.84GB / 6.50GB, 167MB/s  ...eckpoints/model_060000.pt: 75%|███████▍ | 1.76GB / 2.35GB  ...nts/optim_060000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.4GB / 22.2GB, 378MB/s New Data Upload : 90%|█████████ | 5.85GB / 6.50GB, 164MB/s  ...eckpoints/model_060000.pt: 76%|███████▌ | 1.78GB / 2.35GB  ...nts/optim_060000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank6.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▋| 21.4GB / 22.2GB, 366MB/s New Data Upload : 90%|█████████ | 5.88GB / 6.50GB, 162MB/s  ...eckpoints/model_060000.pt: 77%|███████▋ | 1.80GB / 2.35GB  ...nts/optim_060000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank6.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▋| 21.4GB / 22.2GB, 360MB/s New Data Upload : 91%|█████████ | 5.90GB / 6.50GB, 160MB/s  ...eckpoints/model_060000.pt: 77%|███████▋ | 1.82GB / 2.35GB  ...nts/optim_060000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 351MB/s New Data Upload : 91%|█████████ | 5.92GB / 6.50GB, 159MB/s  ...eckpoints/model_060000.pt: 78%|███████▊ | 1.83GB / 2.35GB  ...nts/optim_060000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 343MB/s New Data Upload : 91%|█████████▏| 5.93GB / 6.50GB, 157MB/s  ...eckpoints/model_060000.pt: 79%|███████▊ | 1.85GB / 2.35GB  ...nts/optim_060000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 337MB/s New Data Upload : 92%|█████████▏| 5.95GB / 6.50GB, 155MB/s  ...eckpoints/model_060000.pt: 80%|███████▉ | 1.87GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 331MB/s New Data Upload : 92%|█████████▏| 5.98GB / 6.50GB, 154MB/s  ...eckpoints/model_060000.pt: 80%|████████ | 1.88GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 322MB/s New Data Upload : 92%|█████████▏| 5.99GB / 6.50GB, 151MB/s  ...eckpoints/model_060000.pt: 81%|████████ | 1.90GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 316MB/s New Data Upload : 92%|█████████▏| 6.01GB / 6.50GB, 150MB/s  ...eckpoints/model_060000.pt: 82%|████████▏ | 1.92GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 312MB/s New Data Upload : 93%|█████████▎| 6.03GB / 6.50GB, 149MB/s  ...eckpoints/model_060000.pt: 82%|████████▏ | 1.94GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 303MB/s New Data Upload : 93%|█████████▎| 6.05GB / 6.50GB, 146MB/s  ...eckpoints/model_060000.pt: 83%|████████▎ | 1.95GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.7GB / 22.2GB, 299MB/s New Data Upload : 93%|█████████▎| 6.07GB / 6.50GB, 145MB/s  ...eckpoints/model_060000.pt: 84%|████████▍ | 1.97GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.7GB / 22.2GB, 294MB/s New Data Upload : 94%|█████████▎| 6.09GB / 6.50GB, 144MB/s  ...eckpoints/model_060000.pt: 84%|████████▍ | 1.98GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 288MB/s New Data Upload : 94%|█████████▍| 6.11GB / 6.50GB, 142MB/s  ...eckpoints/model_060000.pt: 85%|████████▌ | 2.00GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 280MB/s New Data Upload : 94%|█████████▍| 6.13GB / 6.50GB, 140MB/s  ...eckpoints/model_060000.pt: 86%|████████▌ | 2.02GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 274MB/s New Data Upload : 95%|█████████▍| 6.15GB / 6.50GB, 139MB/s  ...eckpoints/model_060000.pt: 87%|████████▋ | 2.04GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 269MB/s New Data Upload : 95%|█████████▍| 6.16GB / 6.50GB, 137MB/s  ...eckpoints/model_060000.pt: 87%|████████▋ | 2.05GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 262MB/s New Data Upload : 95%|█████████▌| 6.18GB / 6.50GB, 136MB/s  ...eckpoints/model_060000.pt: 88%|████████▊ | 2.07GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 257MB/s New Data Upload : 95%|█████████▌| 6.20GB / 6.50GB, 135MB/s  ...eckpoints/model_060000.pt: 89%|████████▊ | 2.08GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 252MB/s New Data Upload : 96%|█████████▌| 6.22GB / 6.50GB, 134MB/s  ...eckpoints/model_060000.pt: 89%|████████▉ | 2.10GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 244MB/s New Data Upload : 96%|█████████▌| 6.23GB / 6.50GB, 133MB/s  ...eckpoints/model_060000.pt: 90%|█████████ | 2.11GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 235MB/s New Data Upload : 96%|█████████▌| 6.25GB / 6.50GB, 130MB/s  ...eckpoints/model_060000.pt: 91%|█████████ | 2.13GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 233MB/s New Data Upload : 96%|█████████▋| 6.26GB / 6.50GB, 129MB/s  ...eckpoints/model_060000.pt: 91%|█████████▏| 2.14GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 224MB/s New Data Upload : 97%|█████████▋| 6.28GB / 6.50GB, 127MB/s  ...eckpoints/model_060000.pt: 92%|█████████▏| 2.16GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 218MB/s New Data Upload : 97%|█████████▋| 6.30GB / 6.50GB, 125MB/s  ...eckpoints/model_060000.pt: 93%|█████████▎| 2.18GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 22.0GB / 22.2GB, 212MB/s New Data Upload : 97%|█████████▋| 6.32GB / 6.50GB, 124MB/s  ...eckpoints/model_060000.pt: 93%|█████████▎| 2.19GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 205MB/s New Data Upload : 97%|█████████▋| 6.33GB / 6.50GB, 122MB/s  ...eckpoints/model_060000.pt: 94%|█████████▎| 2.20GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 198MB/s New Data Upload : 97%|█████████▋| 6.34GB / 6.50GB, 119MB/s  ...eckpoints/model_060000.pt: 95%|█████████▍| 2.22GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 193MB/s New Data Upload : 98%|█████████▊| 6.36GB / 6.50GB, 118MB/s  ...eckpoints/model_060000.pt: 95%|█████████▌| 2.23GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 189MB/s New Data Upload : 98%|█████████▊| 6.37GB / 6.50GB, 116MB/s  ...eckpoints/model_060000.pt: 96%|█████████▌| 2.25GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 182MB/s New Data Upload : 98%|█████████▊| 6.39GB / 6.50GB, 113MB/s  ...eckpoints/model_060000.pt: 96%|█████████▌| 2.25GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 176MB/s New Data Upload : 98%|█████████▊| 6.39GB / 6.50GB, 110MB/s  ...eckpoints/model_060000.pt: 96%|█████████▋| 2.26GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 170MB/s New Data Upload : 98%|█████████▊| 6.40GB / 6.50GB, 107MB/s  ...eckpoints/model_060000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_060000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 162MB/s New Data Upload : 99%|█████████▊| 6.41GB / 6.50GB, 104MB/s  ...eckpoints/model_060000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 153MB/s New Data Upload : 99%|█████████▊| 6.42GB / 6.50GB, 99.8MB/s  ...eckpoints/model_060000.pt: 97%|█████████▋| 2.29GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 149MB/s New Data Upload : 99%|█████████▉| 6.43GB / 6.50GB, 98.4MB/s  ...eckpoints/model_060000.pt: 97%|█████████▋| 2.29GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 142MB/s New Data Upload : 99%|█████████▉| 6.43GB / 6.50GB, 95.1MB/s  ...eckpoints/model_060000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 137MB/s New Data Upload : 99%|█████████▉| 6.44GB / 6.50GB, 93.1MB/s  ...eckpoints/model_060000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 131MB/s New Data Upload : 99%|█████████▉| 6.45GB / 6.50GB, 91.2MB/s  ...eckpoints/model_060000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 124MB/s New Data Upload : 99%|█████████▉| 6.46GB / 6.50GB, 88.3MB/s  ...eckpoints/model_060000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 119MB/s New Data Upload : 99%|█████████▉| 6.46GB / 6.50GB, 86.5MB/s  ...eckpoints/model_060000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 114MB/s New Data Upload : 99%|█████████▉| 6.47GB / 6.50GB, 83.8MB/s  ...eckpoints/model_060000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 110MB/s New Data Upload : 100%|█████████▉| 6.47GB / 6.50GB, 81.7MB/s  ...eckpoints/model_060000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 105MB/s New Data Upload : 100%|█████████▉| 6.48GB / 6.50GB, 79.7MB/s  ...eckpoints/model_060000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 101MB/s New Data Upload : 100%|█████████▉| 6.48GB / 6.50GB, 76.7MB/s  ...eckpoints/model_060000.pt: 99%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 97.7MB/s New Data Upload : 100%|█████████▉| 6.49GB / 6.50GB, 74.5MB/s  ...eckpoints/model_060000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 94.9MB/s New Data Upload : 100%|█████████▉| 6.49GB / 6.50GB, 72.4MB/s  ...eckpoints/model_060000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 91.1MB/s New Data Upload : 100%|█████████▉| 6.49GB / 6.50GB, 70.2MB/s  ...eckpoints/model_060000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 88.7MB/s New Data Upload : 100%|█████████▉| 6.49GB / 6.50GB, 68.6MB/s  ...eckpoints/model_060000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 86.7MB/s New Data Upload : 100%|█████████▉| 6.50GB / 6.50GB, 66.7MB/s  ...eckpoints/model_060000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 83.4MB/s New Data Upload : 100%|█████████▉| 6.50GB / 6.50GB, 63.3MB/s  ...eckpoints/model_060000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 78.0MB/s New Data Upload : 100%|█████████▉| 6.50GB / 6.50GB, 59.2MB/s  ...eckpoints/model_060000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 74.6MB/s New Data Upload : 100%|█████████▉| 6.50GB / 6.50GB, 56.8MB/s  ...eckpoints/model_060000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 71.2MB/s New Data Upload : 100%|█████████▉| 6.50GB / 6.50GB, 53.7MB/s  ...eckpoints/model_060000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 66.5MB/s New Data Upload : 100%|█████████▉| 6.50GB / 6.50GB, 49.8MB/s  ...eckpoints/model_060000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (9 / 9) : 100%|██████████| 22.2GB / 22.2GB, 59.8MB/s New Data Upload : 100%|██████████| 6.50GB / 6.50GB, 44.2MB/s  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (9 / 9) : 100%|██████████| 22.2GB / 22.2GB, 25.1MB/s New Data Upload : 100%|██████████| 6.50GB / 6.50GB, 16.3MB/s ...eckpoints/model_060000.pt: 100%|██████████| 2.35GB / 2.35GB ...nts/optim_060000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_060000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_060000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_060000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_060000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_060000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_060000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_060000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB [18:01:13] Step 60000 uploaded [19:01:13] Uploading step 65000... Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...nts/optim_065000_rank0.pt: 0%| | 237kB / 2.49GB  ...nts/optim_065000_rank1.pt: 0%| | 237kB / 2.49GB  ...nts/optim_065000_rank2.pt: 0%| | 237kB / 2.49GB  ...nts/optim_065000_rank4.pt: 0%| | 237kB / 2.49GB  ...nts/optim_065000_rank5.pt: 0%| | 585kB / 2.49GB  ...nts/optim_065000_rank6.pt: 0%| | 237kB / 2.49GB  ...nts/optim_065000_rank7.pt: 0%| | 237kB / 2.49GB  ...nts/optim_065000_rank0.pt: 0%| | 237kB / 2.49GB  ...nts/optim_065000_rank1.pt: 0%| | 237kB / 2.49GB  ...nts/optim_065000_rank2.pt: 0%| | 237kB / 2.49GB  ...nts/optim_065000_rank4.pt: 0%| | 237kB / 2.49GB  ...nts/optim_065000_rank5.pt: 0%| | 585kB / 2.49GB  ...nts/optim_065000_rank6.pt: 0%| | 237kB / 2.49GB  ...nts/optim_065000_rank7.pt: 0%| | 237kB / 2.49GB  Processing Files (0 / 7) : 0%| | 2.01MB / 22.2GB, 2.51MB/s New Data Upload : 0%| | 585kB / 536MB, 731kB/s  ...nts/optim_065000_rank0.pt: 0%| | 237kB / 2.49GB  ...nts/optim_065000_rank1.pt: 0%| | 237kB / 2.49GB  ...nts/optim_065000_rank2.pt: 0%| | 237kB / 2.49GB  ...nts/optim_065000_rank4.pt: 0%| | 237kB / 2.49GB  ...nts/optim_065000_rank5.pt: 0%| | 585kB / 2.49GB  ...nts/optim_065000_rank6.pt: 0%| | 237kB / 2.49GB  ...nts/optim_065000_rank7.pt: 0%| | 237kB / 2.49GB  ...nts/optim_065000_rank0.pt: 0%| | 474kB / 2.49GB  ...nts/optim_065000_rank1.pt: 0%| | 474kB / 2.49GB  ...nts/optim_065000_rank2.pt: 0%| | 474kB / 2.49GB  ...nts/optim_065000_rank4.pt: 0%| | 474kB / 2.49GB  ...nts/optim_065000_rank5.pt: 0%| | 1.17MB / 2.49GB  ...nts/optim_065000_rank6.pt: 0%| | 474kB / 2.49GB  ...nts/optim_065000_rank7.pt: 0%| | 474kB / 2.49GB  Processing Files (0 / 7) : 0%| | 4.01MB / 22.2GB, 3.34MB/s New Data Upload : 0%| | 1.17MB / 604MB, 975kB/s  ...nts/optim_065000_rank0.pt: 0%| | 1.42MB / 2.49GB  ...nts/optim_065000_rank1.pt: 0%| | 1.42MB / 2.49GB  ...nts/optim_065000_rank2.pt: 0%| | 1.42MB / 2.49GB  ...nts/optim_065000_rank4.pt: 0%| | 1.42MB / 2.49GB  ...nts/optim_065000_rank5.pt: 0%| | 3.51MB / 2.49GB  ...nts/optim_065000_rank6.pt: 0%| | 1.42MB / 2.49GB  ...nts/optim_065000_rank7.pt: 0%| | 1.42MB / 2.49GB  Processing Files (0 / 7) : 0%| | 12.0MB / 22.2GB, 8.60MB/s New Data Upload : 1%| | 3.51MB / 604MB, 2.51MB/s  ...nts/optim_065000_rank0.pt: 0%| | 2.13MB / 2.49GB  ...nts/optim_065000_rank1.pt: 0%| | 2.13MB / 2.49GB  ...nts/optim_065000_rank2.pt: 0%| | 2.13MB / 2.49GB  ...nts/optim_065000_rank4.pt: 0%| | 2.13MB / 2.49GB  ...nts/optim_065000_rank5.pt: 0%| | 5.27MB / 2.49GB  ...nts/optim_065000_rank6.pt: 0%| | 2.13MB / 2.49GB  ...nts/optim_065000_rank7.pt: 0%| | 2.13MB / 2.49GB  Processing Files (0 / 7) : 0%| | 18.1MB / 22.2GB, 11.3MB/s New Data Upload : 1%| | 5.27MB / 604MB, 3.29MB/s  ...nts/optim_065000_rank0.pt: 0%| | 3.32MB / 2.49GB  ...nts/optim_065000_rank1.pt: 0%| | 3.32MB / 2.49GB  ...nts/optim_065000_rank2.pt: 0%| | 3.32MB / 2.49GB  ...nts/optim_065000_rank4.pt: 0%| | 3.32MB / 2.49GB  ...nts/optim_065000_rank5.pt: 0%| | 8.19MB / 2.49GB  ...nts/optim_065000_rank6.pt: 0%| | 3.32MB / 2.49GB  ...nts/optim_065000_rank7.pt: 0%| | 3.32MB / 2.49GB  Processing Files (0 / 7) : 0%| | 28.1MB / 22.2GB, 15.6MB/s New Data Upload : 1%|▏ | 8.19MB / 604MB, 4.55MB/s  ...nts/optim_065000_rank0.pt: 0%| | 4.97MB / 2.49GB  ...nts/optim_065000_rank1.pt: 0%| | 4.97MB / 2.49GB  ...nts/optim_065000_rank2.pt: 0%| | 4.97MB / 2.49GB  ...nts/optim_065000_rank4.pt: 0%| | 4.97MB / 2.49GB  ...nts/optim_065000_rank5.pt: 0%| | 12.3MB / 2.49GB  ...nts/optim_065000_rank6.pt: 0%| | 4.97MB / 2.49GB  ...nts/optim_065000_rank7.pt: 0%| | 4.97MB / 2.49GB  Processing Files (0 / 7) : 0%| | 42.1MB / 22.2GB, 21.1MB/s New Data Upload : 2%|▏ | 12.3MB / 604MB, 6.14MB/s  ...nts/optim_065000_rank0.pt: 0%| | 6.87MB / 2.49GB  ...nts/optim_065000_rank1.pt: 0%| | 6.87MB / 2.49GB  ...nts/optim_065000_rank2.pt: 0%| | 6.87MB / 2.49GB  ...nts/optim_065000_rank4.pt: 0%| | 6.87MB / 2.49GB  ...nts/optim_065000_rank5.pt: 1%| | 17.0MB / 2.49GB  ...nts/optim_065000_rank6.pt: 0%| | 6.87MB / 2.49GB  ...nts/optim_065000_rank7.pt: 0%| | 6.87MB / 2.49GB  Processing Files (0 / 7) : 0%| | 58.2MB / 22.2GB, 26.4MB/s New Data Upload : 3%|▎ | 17.0MB / 604MB, 7.71MB/s  ...nts/optim_065000_rank0.pt: 0%| | 7.58MB / 2.49GB  ...nts/optim_065000_rank1.pt: 0%| | 7.58MB / 2.49GB  ...nts/optim_065000_rank2.pt: 0%| | 7.58MB / 2.49GB  ...nts/optim_065000_rank4.pt: 0%| | 7.58MB / 2.49GB  ...nts/optim_065000_rank5.pt: 1%| | 18.7MB / 2.49GB  ...nts/optim_065000_rank6.pt: 0%| | 7.58MB / 2.49GB  ...nts/optim_065000_rank7.pt: 0%| | 7.58MB / 2.49GB  Processing Files (0 / 7) : 0%| | 64.2MB / 22.2GB, 26.8MB/s New Data Upload : 3%|▎ | 18.7MB / 604MB, 7.80MB/s  ...nts/optim_065000_rank0.pt: 0%| | 9.24MB / 2.49GB  ...nts/optim_065000_rank1.pt: 0%| | 9.24MB / 2.49GB  ...nts/optim_065000_rank2.pt: 0%| | 9.24MB / 2.49GB  ...nts/optim_065000_rank4.pt: 0%| | 9.24MB / 2.49GB  ...nts/optim_065000_rank5.pt: 1%| | 22.8MB / 2.49GB  ...nts/optim_065000_rank6.pt: 0%| | 9.24MB / 2.49GB  ...nts/optim_065000_rank7.pt: 0%| | 9.24MB / 2.49GB  Processing Files (0 / 7) : 0%| | 78.2MB / 22.2GB, 30.1MB/s New Data Upload : 4%|▍ | 22.8MB / 604MB, 8.77MB/s  ...nts/optim_065000_rank0.pt: 0%| | 10.9MB / 2.49GB  ...nts/optim_065000_rank1.pt: 0%| | 10.9MB / 2.49GB  ...nts/optim_065000_rank2.pt: 0%| | 10.9MB / 2.49GB  ...nts/optim_065000_rank4.pt: 0%| | 10.9MB / 2.49GB  ...nts/optim_065000_rank5.pt: 1%| | 26.9MB / 2.49GB  ...nts/optim_065000_rank6.pt: 0%| | 10.9MB / 2.49GB  ...nts/optim_065000_rank7.pt: 0%| | 10.9MB / 2.49GB  Processing Files (0 / 7) : 0%| | 92.3MB / 22.2GB, 33.0MB/s New Data Upload : 4%|▍ | 26.9MB / 604MB, 9.61MB/s  ...nts/optim_065000_rank0.pt: 1%| | 12.6MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 12.6MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 12.6MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 12.6MB / 2.49GB  ...nts/optim_065000_rank5.pt: 1%| | 31.0MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 12.6MB / 2.49GB  ...nts/optim_065000_rank7.pt: 1%| | 12.6MB / 2.49GB  Processing Files (0 / 7) : 0%| | 106MB / 22.2GB, 35.4MB/s New Data Upload : 5%|▌ | 31.0MB / 604MB, 10.3MB/s  ...nts/optim_065000_rank0.pt: 1%| | 14.0MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 14.0MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 14.0MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 14.0MB / 2.49GB  ...nts/optim_065000_rank5.pt: 1%|▏ | 34.5MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 14.0MB / 2.49GB  ...nts/optim_065000_rank7.pt: 1%| | 14.0MB / 2.49GB  Processing Files (0 / 7) : 1%| | 118MB / 22.2GB, 37.0MB/s New Data Upload : 6%|▌ | 34.5MB / 604MB, 10.8MB/s  ...nts/optim_065000_rank0.pt: 1%| | 15.6MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 15.6MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 15.6MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 15.6MB / 2.49GB  ...nts/optim_065000_rank5.pt: 2%|▏ | 38.6MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 15.6MB / 2.49GB  ...nts/optim_065000_rank7.pt: 1%| | 15.6MB / 2.49GB  Processing Files (0 / 7) : 1%| | 132MB / 22.2GB, 38.9MB/s New Data Upload : 6%|▋ | 38.6MB / 604MB, 11.4MB/s  ...nts/optim_065000_rank0.pt: 1%| | 17.5MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 17.5MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 17.5MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 17.5MB / 2.49GB  ...nts/optim_065000_rank5.pt: 2%|▏ | 43.3MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 17.5MB / 2.49GB  ...nts/optim_065000_rank7.pt: 1%| | 17.5MB / 2.49GB  Processing Files (0 / 7) : 1%| | 148MB / 22.2GB, 41.2MB/s New Data Upload : 7%|▋ | 43.3MB / 604MB, 12.0MB/s  ...nts/optim_065000_rank0.pt: 1%| | 18.2MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 18.2MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 18.2MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 18.2MB / 2.49GB  ...nts/optim_065000_rank5.pt: 2%|▏ | 45.0MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 18.2MB / 2.49GB  ...nts/optim_065000_rank7.pt: 1%| | 18.2MB / 2.49GB  Processing Files (0 / 7) : 1%| | 154MB / 22.2GB, 40.6MB/s New Data Upload : 7%|▋ | 45.0MB / 604MB, 11.9MB/s  ...nts/optim_065000_rank0.pt: 1%| | 19.9MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 19.9MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 19.9MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 19.9MB / 2.49GB  ...nts/optim_065000_rank5.pt: 2%|▏ | 49.1MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 19.9MB / 2.49GB  ...nts/optim_065000_rank7.pt: 1%| | 19.9MB / 2.49GB  Processing Files (0 / 7) : 1%| | 169MB / 22.2GB, 42.1MB/s New Data Upload : 8%|▊ | 49.1MB / 604MB, 12.3MB/s  ...nts/optim_065000_rank0.pt: 1%| | 21.6MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 21.6MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 21.6MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 21.6MB / 2.49GB  ...nts/optim_065000_rank5.pt: 2%|▏ | 53.2MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 21.6MB / 2.49GB  ...nts/optim_065000_rank7.pt: 1%| | 21.6MB / 2.49GB  Processing Files (0 / 7) : 1%| | 183MB / 22.2GB, 43.5MB/s New Data Upload : 9%|▉ | 53.2MB / 604MB, 12.7MB/s  ...nts/optim_065000_rank0.pt: 1%| | 23.4MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 23.4MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 23.4MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 23.4MB / 2.49GB  ...nts/optim_065000_rank5.pt: 2%|▏ | 57.9MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 23.4MB / 2.49GB  ...nts/optim_065000_rank7.pt: 1%| | 23.4MB / 2.49GB  Processing Files (0 / 7) : 1%| | 199MB / 22.2GB, 45.1MB/s New Data Upload : 10%|▉ | 57.9MB / 604MB, 13.2MB/s  ...nts/optim_065000_rank0.pt: 1%| | 25.1MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 25.1MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 25.1MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 25.1MB / 2.49GB  ...nts/optim_065000_rank5.pt: 2%|▏ | 62.0MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 25.1MB / 2.49GB  ...nts/optim_065000_rank7.pt: 1%| | 25.1MB / 2.49GB  Processing Files (0 / 7) : 1%| | 213MB / 22.2GB, 46.2MB/s New Data Upload : 10%|█ | 62.0MB / 604MB, 13.5MB/s  ...nts/optim_065000_rank0.pt: 1%| | 26.8MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 26.8MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 26.8MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 26.8MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 66.1MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 26.8MB / 2.49GB  ...nts/optim_065000_rank7.pt: 1%| | 26.8MB / 2.49GB  Processing Files (0 / 7) : 1%| | 227MB / 22.2GB, 47.2MB/s New Data Upload : 11%|█ | 66.1MB / 604MB, 13.8MB/s  ...nts/optim_065000_rank0.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 66.7MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank7.pt: 1%| | 27.0MB / 2.49GB  Processing Files (0 / 7) : 1%| | 229MB / 22.2GB, 45.7MB/s New Data Upload : 11%|█ | 66.7MB / 604MB, 13.3MB/s  ...nts/optim_065000_rank0.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 66.7MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank7.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank0.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 66.7MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank7.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank0.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 66.7MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank7.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_065000_rank0.pt: 1%| | 27.2MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 27.2MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 27.2MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 27.2MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 27.2MB / 2.49GB  ...nts/optim_065000_rank7.pt: 1%| | 27.2MB / 2.49GB  Processing Files (0 / 7) : 1%| | 230MB / 22.2GB, 39.7MB/s New Data Upload : 11%|█ | 67.1MB / 604MB, 11.6MB/s  ...eckpoints/model_065000.pt: 0%| | 581kB / 2.35GB  ...nts/optim_065000_rank0.pt: 1%| | 27.5MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 27.8MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 27.5MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 28.0MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 67.5MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 28.0MB / 2.49GB  ...nts/optim_065000_rank7.pt: 1%|▏ | 31.3MB / 2.49GB  ...eckpoints/model_065000.pt: 0%| | 581kB / 2.35GB  Processing Files (0 / 8) : 1%| | 238MB / 22.2GB, 39.7MB/s New Data Upload : 12%|█▏ | 71.8MB / 604MB, 12.0MB/s  ...nts/optim_065000_rank0.pt: 1%| | 27.9MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 28.4MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 27.9MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 28.8MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 67.8MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 28.8MB / 2.49GB  ...nts/optim_065000_rank7.pt: 1%|▏ | 34.8MB / 2.49GB  ...eckpoints/model_065000.pt: 0%| | 3.66MB / 2.35GB  Processing Files (0 / 8) : 1%| | 248MB / 22.2GB, 40.0MB/s New Data Upload : 10%|█ | 75.3MB / 738MB, 12.1MB/s  ...nts/optim_065000_rank0.pt: 1%| | 28.3MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 29.2MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 28.3MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 29.8MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 68.2MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 29.7MB / 2.49GB  ...nts/optim_065000_rank7.pt: 2%|▏ | 39.4MB / 2.49GB  ...eckpoints/model_065000.pt: 0%| | 4.82MB / 2.35GB  Processing Files (0 / 8) : 1%| | 258MB / 22.2GB, 40.3MB/s New Data Upload : 11%|█ | 81.1MB / 738MB, 12.7MB/s  ...nts/optim_065000_rank0.pt: 1%| | 28.5MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 29.6MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 29.0MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%| | 30.2MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 68.4MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%| | 30.2MB / 2.49GB  ...nts/optim_065000_rank7.pt: 2%|▏ | 41.2MB / 2.49GB  ...eckpoints/model_065000.pt: 0%| | 9.47MB / 2.35GB  Processing Files (0 / 8) : 1%| | 267MB / 22.2GB, 40.4MB/s New Data Upload : 12%|█▏ | 88.1MB / 738MB, 13.3MB/s  ...nts/optim_065000_rank0.pt: 1%| | 28.9MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%| | 30.4MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%| | 29.5MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%|▏ | 31.2MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 68.9MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%|▏ | 31.2MB / 2.49GB  ...nts/optim_065000_rank7.pt: 2%|▏ | 45.9MB / 2.49GB  ...eckpoints/model_065000.pt: 1%| | 15.3MB / 2.35GB  Processing Files (0 / 8) : 1%|▏ | 281MB / 22.2GB, 41.4MB/s New Data Upload : 12%|█▏ | 98.6MB / 805MB, 14.5MB/s  ...nts/optim_065000_rank0.pt: 1%| | 30.1MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%|▏ | 32.0MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%|▏ | 32.2MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%|▏ | 32.5MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 69.6MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%|▏ | 32.8MB / 2.49GB  ...nts/optim_065000_rank7.pt: 2%|▏ | 50.8MB / 2.49GB  ...eckpoints/model_065000.pt: 1%| | 24.0MB / 2.35GB  Processing Files (0 / 8) : 1%|▏ | 304MB / 22.2GB, 43.4MB/s New Data Upload : 14%|█▍ | 115MB / 805MB, 16.4MB/s  ...nts/optim_065000_rank0.pt: 1%| | 30.6MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%|▏ | 33.2MB / 2.49GB  ...nts/optim_065000_rank2.pt: 1%|▏ | 34.4MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%|▏ | 33.5MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 70.1MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%|▏ | 34.0MB / 2.49GB  ...nts/optim_065000_rank7.pt: 2%|▏ | 55.0MB / 2.49GB  ...eckpoints/model_065000.pt: 1%|▏ | 32.7MB / 2.35GB  Processing Files (0 / 8) : 1%|▏ | 323MB / 22.2GB, 44.9MB/s New Data Upload : 15%|█▍ | 129MB / 872MB, 18.0MB/s  ...nts/optim_065000_rank0.pt: 1%|▏ | 32.1MB / 2.49GB  ...nts/optim_065000_rank1.pt: 1%|▏ | 36.2MB / 2.49GB  ...nts/optim_065000_rank2.pt: 2%|▏ | 42.4MB / 2.49GB  ...nts/optim_065000_rank4.pt: 1%|▏ | 35.1MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 71.2MB / 2.49GB  ...nts/optim_065000_rank6.pt: 1%|▏ | 36.5MB / 2.49GB  ...nts/optim_065000_rank7.pt: 2%|▏ | 59.6MB / 2.49GB  ...eckpoints/model_065000.pt: 2%|▏ | 37.3MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 350MB / 22.2GB, 47.4MB/s New Data Upload : 17%|█▋ | 146MB / 872MB, 19.8MB/s  ...nts/optim_065000_rank0.pt: 1%|▏ | 37.1MB / 2.49GB  ...nts/optim_065000_rank1.pt: 2%|▏ | 40.1MB / 2.49GB  ...nts/optim_065000_rank2.pt: 2%|▏ | 52.1MB / 2.49GB  ...nts/optim_065000_rank4.pt: 2%|▏ | 37.5MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 73.5MB / 2.49GB  ...nts/optim_065000_rank6.pt: 2%|▏ | 39.5MB / 2.49GB  ...nts/optim_065000_rank7.pt: 3%|▎ | 63.5MB / 2.49GB  ...eckpoints/model_065000.pt: 2%|▏ | 37.3MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 381MB / 22.2GB, 50.1MB/s New Data Upload : 19%|█▊ | 163MB / 872MB, 21.5MB/s  ...nts/optim_065000_rank0.pt: 2%|▏ | 40.8MB / 2.49GB  ...nts/optim_065000_rank1.pt: 2%|▏ | 43.3MB / 2.49GB  ...nts/optim_065000_rank2.pt: 2%|▏ | 61.1MB / 2.49GB  ...nts/optim_065000_rank4.pt: 2%|▏ | 38.8MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 75.3MB / 2.49GB  ...nts/optim_065000_rank6.pt: 2%|▏ | 41.9MB / 2.49GB  ...nts/optim_065000_rank7.pt: 3%|▎ | 66.1MB / 2.49GB  ...eckpoints/model_065000.pt: 2%|▏ | 41.4MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 409MB / 22.2GB, 52.4MB/s New Data Upload : 21%|██ | 181MB / 872MB, 23.2MB/s  ...nts/optim_065000_rank0.pt: 2%|▏ | 45.7MB / 2.49GB  ...nts/optim_065000_rank1.pt: 2%|▏ | 46.0MB / 2.49GB  ...nts/optim_065000_rank2.pt: 3%|▎ | 66.3MB / 2.49GB  ...nts/optim_065000_rank4.pt: 2%|▏ | 41.5MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 77.4MB / 2.49GB  ...nts/optim_065000_rank6.pt: 2%|▏ | 44.1MB / 2.49GB  ...nts/optim_065000_rank7.pt: 3%|▎ | 70.3MB / 2.49GB  ...eckpoints/model_065000.pt: 2%|▏ | 46.0MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 437MB / 22.2GB, 54.7MB/s New Data Upload : 23%|██▎ | 199MB / 872MB, 24.9MB/s  ...nts/optim_065000_rank0.pt: 2%|▏ | 51.8MB / 2.49GB  ...nts/optim_065000_rank1.pt: 2%|▏ | 50.8MB / 2.49GB  ...nts/optim_065000_rank2.pt: 3%|▎ | 76.1MB / 2.49GB  ...nts/optim_065000_rank4.pt: 2%|▏ | 48.1MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 80.6MB / 2.49GB  ...nts/optim_065000_rank6.pt: 2%|▏ | 48.1MB / 2.49GB  ...nts/optim_065000_rank7.pt: 3%|▎ | 75.4MB / 2.49GB  ...eckpoints/model_065000.pt: 2%|▏ | 46.0MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 477MB / 22.2GB, 58.2MB/s New Data Upload : 24%|██▎ | 221MB / 939MB, 27.0MB/s  ...nts/optim_065000_rank0.pt: 2%|▏ | 60.6MB / 2.49GB  ...nts/optim_065000_rank1.pt: 2%|▏ | 55.3MB / 2.49GB  ...nts/optim_065000_rank2.pt: 3%|▎ | 86.2MB / 2.49GB  ...nts/optim_065000_rank4.pt: 2%|▏ | 52.0MB / 2.49GB  ...nts/optim_065000_rank5.pt: 3%|▎ | 84.5MB / 2.49GB  ...nts/optim_065000_rank6.pt: 2%|▏ | 52.2MB / 2.49GB  ...nts/optim_065000_rank7.pt: 3%|▎ | 80.3MB / 2.49GB  ...eckpoints/model_065000.pt: 2%|▏ | 46.0MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 517MB / 22.2GB, 61.6MB/s New Data Upload : 26%|██▌ | 244MB / 939MB, 29.0MB/s  ...nts/optim_065000_rank0.pt: 3%|▎ | 66.1MB / 2.49GB  ...nts/optim_065000_rank1.pt: 2%|▏ | 58.2MB / 2.49GB  ...nts/optim_065000_rank2.pt: 4%|▎ | 87.7MB / 2.49GB  ...nts/optim_065000_rank4.pt: 2%|▏ | 61.9MB / 2.49GB  ...nts/optim_065000_rank5.pt: 4%|▎ | 87.3MB / 2.49GB  ...nts/optim_065000_rank6.pt: 2%|▏ | 54.7MB / 2.49GB  ...nts/optim_065000_rank7.pt: 3%|▎ | 85.2MB / 2.49GB  ...eckpoints/model_065000.pt: 2%|▏ | 46.0MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 547MB / 22.2GB, 63.6MB/s New Data Upload : 26%|██▌ | 261MB / 1.01GB, 30.3MB/s  ...nts/optim_065000_rank0.pt: 3%|▎ | 76.8MB / 2.49GB  ...nts/optim_065000_rank1.pt: 3%|▎ | 62.6MB / 2.49GB  ...nts/optim_065000_rank2.pt: 4%|▎ | 90.2MB / 2.49GB  ...nts/optim_065000_rank4.pt: 3%|▎ | 72.6MB / 2.49GB  ...nts/optim_065000_rank5.pt: 4%|▎ | 93.2MB / 2.49GB  ...nts/optim_065000_rank6.pt: 2%|▏ | 61.9MB / 2.49GB  ...nts/optim_065000_rank7.pt: 4%|▎ | 91.4MB / 2.49GB  ...eckpoints/model_065000.pt: 2%|▏ | 46.0MB / 2.35GB  Processing Files (0 / 8) : 3%|▎ | 595MB / 22.2GB, 67.6MB/s New Data Upload : 29%|██▊ | 287MB / 1.01GB, 32.7MB/s  ...nts/optim_065000_rank0.pt: 3%|▎ | 80.7MB / 2.49GB  ...nts/optim_065000_rank1.pt: 3%|▎ | 65.9MB / 2.49GB  ...nts/optim_065000_rank2.pt: 4%|▎ | 92.4MB / 2.49GB  ...nts/optim_065000_rank4.pt: 3%|▎ | 82.7MB / 2.49GB  ...nts/optim_065000_rank5.pt: 4%|▍ | 96.6MB / 2.49GB  ...nts/optim_065000_rank6.pt: 3%|▎ | 68.0MB / 2.49GB  ...nts/optim_065000_rank7.pt: 4%|▍ | 93.7MB / 2.49GB  ...eckpoints/model_065000.pt: 2%|▏ | 46.0MB / 2.35GB  Processing Files (0 / 8) : 3%|▎ | 626MB / 22.2GB, 69.6MB/s New Data Upload : 30%|███ | 305MB / 1.01GB, 33.8MB/s  ...nts/optim_065000_rank0.pt: 4%|▎ | 91.1MB / 2.49GB  ...nts/optim_065000_rank1.pt: 3%|▎ | 73.8MB / 2.49GB  ...nts/optim_065000_rank2.pt: 4%|▍ | 95.6MB / 2.49GB  ...nts/optim_065000_rank4.pt: 4%|▍ | 94.3MB / 2.49GB  ...nts/optim_065000_rank5.pt: 4%|▍ | 102MB / 2.49GB  ...nts/optim_065000_rank6.pt: 3%|▎ | 73.4MB / 2.49GB  ...nts/optim_065000_rank7.pt: 4%|▍ | 101MB / 2.49GB  ...eckpoints/model_065000.pt: 2%|▏ | 47.8MB / 2.35GB  Processing Files (0 / 8) : 3%|▎ | 679MB / 22.2GB, 73.9MB/s New Data Upload : 31%|███ | 334MB / 1.07GB, 36.3MB/s  ...nts/optim_065000_rank0.pt: 4%|▍ | 100MB / 2.49GB  ...nts/optim_065000_rank1.pt: 3%|▎ | 81.4MB / 2.49GB  ...nts/optim_065000_rank2.pt: 4%|▍ | 104MB / 2.49GB  ...nts/optim_065000_rank4.pt: 4%|▍ | 102MB / 2.49GB  ...nts/optim_065000_rank5.pt: 4%|▍ | 108MB / 2.49GB  ...nts/optim_065000_rank6.pt: 3%|▎ | 84.0MB / 2.49GB  ...nts/optim_065000_rank7.pt: 4%|▍ | 107MB / 2.49GB  ...eckpoints/model_065000.pt: 2%|▏ | 49.5MB / 2.35GB  Processing Files (0 / 8) : 3%|▎ | 737MB / 22.2GB, 78.4MB/s New Data Upload : 34%|███▍ | 364MB / 1.07GB, 38.7MB/s  ...nts/optim_065000_rank0.pt: 4%|▍ | 110MB / 2.49GB  ...nts/optim_065000_rank1.pt: 3%|▎ | 85.3MB / 2.49GB  ...nts/optim_065000_rank2.pt: 4%|▍ | 107MB / 2.49GB  ...nts/optim_065000_rank4.pt: 5%|▍ | 113MB / 2.49GB  ...nts/optim_065000_rank5.pt: 5%|▍ | 114MB / 2.49GB  ...nts/optim_065000_rank6.pt: 4%|▎ | 91.5MB / 2.49GB  ...nts/optim_065000_rank7.pt: 5%|▍ | 114MB / 2.49GB  ...eckpoints/model_065000.pt: 2%|▏ | 50.1MB / 2.35GB  Processing Files (0 / 8) : 4%|▎ | 784MB / 22.2GB, 81.7MB/s New Data Upload : 34%|███▍ | 389MB / 1.14GB, 40.5MB/s  ...nts/optim_065000_rank0.pt: 5%|▍ | 122MB / 2.49GB  ...nts/optim_065000_rank1.pt: 4%|▍ | 97.0MB / 2.49GB  ...nts/optim_065000_rank2.pt: 5%|▍ | 117MB / 2.49GB  ...nts/optim_065000_rank4.pt: 5%|▌ | 128MB / 2.49GB  ...nts/optim_065000_rank5.pt: 5%|▍ | 124MB / 2.49GB  ...nts/optim_065000_rank6.pt: 4%|▍ | 106MB / 2.49GB  ...nts/optim_065000_rank7.pt: 5%|▍ | 122MB / 2.49GB  ...eckpoints/model_065000.pt: 2%|▏ | 58.2MB / 2.35GB  Processing Files (0 / 8) : 4%|▍ | 875MB / 22.2GB, 89.3MB/s New Data Upload : 36%|███▌ | 433MB / 1.21GB, 44.2MB/s  ...nts/optim_065000_rank0.pt: 5%|▌ | 127MB / 2.49GB  ...nts/optim_065000_rank1.pt: 4%|▍ | 102MB / 2.49GB  ...nts/optim_065000_rank2.pt: 5%|▌ | 127MB / 2.49GB  ...nts/optim_065000_rank4.pt: 5%|▌ | 134MB / 2.49GB  ...nts/optim_065000_rank5.pt: 5%|▌ | 130MB / 2.49GB  ...nts/optim_065000_rank6.pt: 5%|▍ | 116MB / 2.49GB  ...nts/optim_065000_rank7.pt: 5%|▌ | 128MB / 2.49GB  ...eckpoints/model_065000.pt: 2%|▏ | 58.2MB / 2.35GB  Processing Files (0 / 8) : 4%|▍ | 922MB / 22.2GB, 92.2MB/s New Data Upload : 37%|███▋ | 448MB / 1.21GB, 44.8MB/s  ...nts/optim_065000_rank0.pt: 5%|▌ | 131MB / 2.49GB  ...nts/optim_065000_rank1.pt: 5%|▍ | 112MB / 2.49GB  ...nts/optim_065000_rank2.pt: 5%|▌ | 133MB / 2.49GB  ...nts/optim_065000_rank4.pt: 6%|▌ | 138MB / 2.49GB  ...nts/optim_065000_rank5.pt: 5%|▌ | 135MB / 2.49GB  ...nts/optim_065000_rank6.pt: 5%|▌ | 126MB / 2.49GB  ...nts/optim_065000_rank7.pt: 5%|▌ | 135MB / 2.49GB  ...eckpoints/model_065000.pt: 3%|▎ | 65.2MB / 2.35GB  Processing Files (0 / 8) : 4%|▍ | 975MB / 22.2GB, 95.6MB/s New Data Upload : 39%|███▉ | 473MB / 1.21GB, 46.4MB/s  ...nts/optim_065000_rank0.pt: 5%|▌ | 135MB / 2.49GB  ...nts/optim_065000_rank1.pt: 5%|▍ | 118MB / 2.49GB  ...nts/optim_065000_rank2.pt: 5%|▌ | 137MB / 2.49GB  ...nts/optim_065000_rank4.pt: 6%|▌ | 141MB / 2.49GB  ...nts/optim_065000_rank5.pt: 6%|▌ | 140MB / 2.49GB  ...nts/optim_065000_rank6.pt: 5%|▌ | 132MB / 2.49GB  ...nts/optim_065000_rank7.pt: 6%|▌ | 138MB / 2.49GB  ...eckpoints/model_065000.pt: 3%|▎ | 74.4MB / 2.35GB  Processing Files (0 / 8) : 5%|▍ | 1.02GB / 22.2GB, 99.5MB/s New Data Upload : 41%|████ | 493MB / 1.21GB, 48.3MB/s  ...nts/optim_065000_rank0.pt: 6%|▌ | 139MB / 2.49GB  ...nts/optim_065000_rank1.pt: 5%|▍ | 124MB / 2.49GB  ...nts/optim_065000_rank2.pt: 6%|▌ | 141MB / 2.49GB  ...nts/optim_065000_rank4.pt: 6%|▌ | 145MB / 2.49GB  ...nts/optim_065000_rank5.pt: 6%|▌ | 144MB / 2.49GB  ...nts/optim_065000_rank6.pt: 6%|▌ | 141MB / 2.49GB  ...nts/optim_065000_rank7.pt: 6%|▌ | 143MB / 2.49GB  ...eckpoints/model_065000.pt: 3%|▎ | 81.2MB / 2.35GB  Processing Files (0 / 8) : 5%|▍ | 1.06GB / 22.2GB, 104MB/s New Data Upload : 42%|████▏ | 513MB / 1.21GB, 50.3MB/s  ...nts/optim_065000_rank0.pt: 6%|▌ | 150MB / 2.49GB  ...nts/optim_065000_rank1.pt: 6%|▌ | 139MB / 2.49GB  ...nts/optim_065000_rank2.pt: 6%|▌ | 154MB / 2.49GB  ...nts/optim_065000_rank4.pt: 6%|▋ | 157MB / 2.49GB  ...nts/optim_065000_rank5.pt: 6%|▋ | 157MB / 2.49GB  ...nts/optim_065000_rank6.pt: 6%|▋ | 159MB / 2.49GB  ...nts/optim_065000_rank7.pt: 6%|▌ | 155MB / 2.49GB  ...eckpoints/model_065000.pt: 4%|▎ | 85.8MB / 2.35GB  Processing Files (0 / 8) : 5%|▌ | 1.16GB / 22.2GB, 113MB/s New Data Upload : 45%|████▍ | 541MB / 1.21GB, 53.0MB/s  ...nts/optim_065000_rank0.pt: 6%|▋ | 158MB / 2.49GB  ...nts/optim_065000_rank1.pt: 6%|▌ | 150MB / 2.49GB  ...nts/optim_065000_rank2.pt: 7%|▋ | 162MB / 2.49GB  ...nts/optim_065000_rank4.pt: 7%|▋ | 165MB / 2.49GB  ...nts/optim_065000_rank5.pt: 7%|▋ | 165MB / 2.49GB  ...nts/optim_065000_rank6.pt: 7%|▋ | 168MB / 2.49GB  ...nts/optim_065000_rank7.pt: 7%|▋ | 163MB / 2.49GB  ...eckpoints/model_065000.pt: 4%|▍ | 94.3MB / 2.35GB  Processing Files (0 / 8) : 6%|▌ | 1.23GB / 22.2GB, 120MB/s New Data Upload : 44%|████▍ | 563MB / 1.27GB, 55.1MB/s  ...nts/optim_065000_rank0.pt: 7%|▋ | 178MB / 2.49GB  ...nts/optim_065000_rank1.pt: 7%|▋ | 171MB / 2.49GB  ...nts/optim_065000_rank2.pt: 7%|▋ | 183MB / 2.49GB  ...nts/optim_065000_rank4.pt: 7%|▋ | 185MB / 2.49GB  ...nts/optim_065000_rank5.pt: 7%|▋ | 185MB / 2.49GB  ...nts/optim_065000_rank6.pt: 8%|▊ | 188MB / 2.49GB  ...nts/optim_065000_rank7.pt: 7%|▋ | 183MB / 2.49GB  ...eckpoints/model_065000.pt: 4%|▍ | 103MB / 2.35GB  Processing Files (0 / 8) : 6%|▌ | 1.38GB / 22.2GB, 135MB/s New Data Upload : 44%|████▍ | 595MB / 1.34GB, 58.2MB/s  ...nts/optim_065000_rank0.pt: 8%|▊ | 200MB / 2.49GB  ...nts/optim_065000_rank1.pt: 8%|▊ | 196MB / 2.49GB  ...nts/optim_065000_rank2.pt: 8%|▊ | 204MB / 2.49GB  ...nts/optim_065000_rank4.pt: 8%|▊ | 206MB / 2.49GB  ...nts/optim_065000_rank5.pt: 8%|▊ | 205MB / 2.49GB  ...nts/optim_065000_rank6.pt: 8%|▊ | 209MB / 2.49GB  ...nts/optim_065000_rank7.pt: 8%|▊ | 203MB / 2.49GB  ...eckpoints/model_065000.pt: 5%|▍ | 112MB / 2.35GB  Processing Files (0 / 8) : 7%|▋ | 1.53GB / 22.2GB, 150MB/s New Data Upload : 47%|████▋ | 630MB / 1.34GB, 61.6MB/s  ...nts/optim_065000_rank0.pt: 9%|▉ | 218MB / 2.49GB  ...nts/optim_065000_rank1.pt: 9%|▊ | 217MB / 2.49GB  ...nts/optim_065000_rank2.pt: 9%|▉ | 226MB / 2.49GB  ...nts/optim_065000_rank4.pt: 9%|▉ | 224MB / 2.49GB  ...nts/optim_065000_rank5.pt: 9%|▉ | 223MB / 2.49GB  ...nts/optim_065000_rank6.pt: 9%|▉ | 227MB / 2.49GB  ...nts/optim_065000_rank7.pt: 9%|▉ | 222MB / 2.49GB  ...eckpoints/model_065000.pt: 5%|▌ | 121MB / 2.35GB  Processing Files (0 / 8) : 8%|▊ | 1.68GB / 22.2GB, 163MB/s New Data Upload : 43%|████▎ | 664MB / 1.54GB, 64.8MB/s  ...nts/optim_065000_rank0.pt: 9%|▉ | 235MB / 2.49GB  ...nts/optim_065000_rank1.pt: 10%|▉ | 237MB / 2.49GB  ...nts/optim_065000_rank2.pt: 10%|▉ | 243MB / 2.49GB  ...nts/optim_065000_rank4.pt: 10%|▉ | 241MB / 2.49GB  ...nts/optim_065000_rank5.pt: 10%|▉ | 239MB / 2.49GB  ...nts/optim_065000_rank6.pt: 10%|▉ | 243MB / 2.49GB  ...nts/optim_065000_rank7.pt: 10%|▉ | 237MB / 2.49GB  ...eckpoints/model_065000.pt: 5%|▌ | 121MB / 2.35GB  Processing Files (0 / 8) : 8%|▊ | 1.79GB / 22.2GB, 174MB/s New Data Upload : 39%|███▉ | 685MB / 1.74GB, 66.7MB/s  ...nts/optim_065000_rank0.pt: 10%|█ | 258MB / 2.49GB  ...nts/optim_065000_rank1.pt: 11%|█ | 262MB / 2.49GB  ...nts/optim_065000_rank2.pt: 11%|█ | 265MB / 2.49GB  ...nts/optim_065000_rank4.pt: 11%|█ | 264MB / 2.49GB  ...nts/optim_065000_rank5.pt: 11%|█ | 262MB / 2.49GB  ...nts/optim_065000_rank6.pt: 11%|█ | 266MB / 2.49GB  ...nts/optim_065000_rank7.pt: 10%|█ | 261MB / 2.49GB  ...eckpoints/model_065000.pt: 6%|▌ | 131MB / 2.35GB  Processing Files (0 / 8) : 9%|▉ | 1.97GB / 22.2GB, 190MB/s New Data Upload : 40%|███▉ | 724MB / 1.81GB, 70.1MB/s  ...nts/optim_065000_rank0.pt: 11%|█ | 280MB / 2.49GB  ...nts/optim_065000_rank1.pt: 12%|█▏ | 286MB / 2.49GB  ...nts/optim_065000_rank2.pt: 12%|█▏ | 289MB / 2.49GB  ...nts/optim_065000_rank4.pt: 11%|█▏ | 285MB / 2.49GB  ...nts/optim_065000_rank5.pt: 11%|█▏ | 282MB / 2.49GB  ...nts/optim_065000_rank6.pt: 12%|█▏ | 286MB / 2.49GB  ...nts/optim_065000_rank7.pt: 11%|█▏ | 281MB / 2.49GB  ...eckpoints/model_065000.pt: 6%|▌ | 137MB / 2.35GB  Processing Files (0 / 8) : 10%|▉ | 2.13GB / 22.2GB, 204MB/s New Data Upload : 40%|████ | 759MB / 1.88GB, 73.2MB/s  ...nts/optim_065000_rank0.pt: 12%|█▏ | 303MB / 2.49GB  ...nts/optim_065000_rank1.pt: 13%|█▎ | 311MB / 2.49GB  ...nts/optim_065000_rank2.pt: 12%|█▏ | 309MB / 2.49GB  ...nts/optim_065000_rank4.pt: 12%|█▏ | 308MB / 2.49GB  ...nts/optim_065000_rank5.pt: 12%|█▏ | 305MB / 2.49GB  ...nts/optim_065000_rank6.pt: 12%|█▏ | 310MB / 2.49GB  ...nts/optim_065000_rank7.pt: 12%|█▏ | 305MB / 2.49GB  ...eckpoints/model_065000.pt: 6%|▌ | 140MB / 2.35GB  Processing Files (0 / 8) : 10%|█ | 2.29GB / 22.2GB, 219MB/s New Data Upload : 41%|████ | 788MB / 1.94GB, 75.6MB/s  ...nts/optim_065000_rank0.pt: 13%|█▎ | 327MB / 2.49GB  ...nts/optim_065000_rank1.pt: 13%|█▎ | 335MB / 2.49GB  ...nts/optim_065000_rank2.pt: 13%|█▎ | 334MB / 2.49GB  ...nts/optim_065000_rank4.pt: 13%|█▎ | 332MB / 2.49GB  ...nts/optim_065000_rank5.pt: 13%|█▎ | 329MB / 2.49GB  ...nts/optim_065000_rank6.pt: 13%|█▎ | 334MB / 2.49GB  ...nts/optim_065000_rank7.pt: 13%|█▎ | 329MB / 2.49GB  ...eckpoints/model_065000.pt: 6%|▋ | 150MB / 2.35GB  Processing Files (0 / 8) : 11%|█ | 2.47GB / 22.2GB, 236MB/s New Data Upload : 43%|████▎ | 829MB / 1.94GB, 79.5MB/s  ...nts/optim_065000_rank0.pt: 14%|█▍ | 355MB / 2.49GB  ...nts/optim_065000_rank1.pt: 15%|█▍ | 362MB / 2.49GB  ...nts/optim_065000_rank2.pt: 14%|█▍ | 357MB / 2.49GB  ...nts/optim_065000_rank4.pt: 14%|█▍ | 359MB / 2.49GB  ...nts/optim_065000_rank5.pt: 14%|█▍ | 356MB / 2.49GB  ...nts/optim_065000_rank6.pt: 15%|█▍ | 361MB / 2.49GB  ...nts/optim_065000_rank7.pt: 14%|█▍ | 356MB / 2.49GB  ...eckpoints/model_065000.pt: 7%|▋ | 156MB / 2.35GB  Processing Files (0 / 8) : 12%|█▏ | 2.66GB / 22.2GB, 253MB/s New Data Upload : 44%|████▍ | 864MB / 1.94GB, 82.5MB/s  ...nts/optim_065000_rank0.pt: 15%|█▌ | 381MB / 2.49GB  ...nts/optim_065000_rank1.pt: 16%|█▌ | 388MB / 2.49GB  ...nts/optim_065000_rank2.pt: 15%|█▌ | 383MB / 2.49GB  ...nts/optim_065000_rank4.pt: 15%|█▌ | 385MB / 2.49GB  ...nts/optim_065000_rank5.pt: 15%|█▌ | 381MB / 2.49GB  ...nts/optim_065000_rank6.pt: 16%|█▌ | 387MB / 2.49GB  ...nts/optim_065000_rank7.pt: 15%|█▌ | 382MB / 2.49GB  ...eckpoints/model_065000.pt: 7%|▋ | 166MB / 2.35GB  Processing Files (0 / 8) : 13%|█▎ | 2.85GB / 22.2GB, 271MB/s New Data Upload : 45%|████▍ | 905MB / 2.01GB, 86.1MB/s  ...nts/optim_065000_rank0.pt: 16%|█▌ | 400MB / 2.49GB  ...nts/optim_065000_rank1.pt: 16%|█▋ | 406MB / 2.49GB  ...nts/optim_065000_rank2.pt: 16%|█▌ | 400MB / 2.49GB  ...nts/optim_065000_rank4.pt: 16%|█▌ | 403MB / 2.49GB  ...nts/optim_065000_rank5.pt: 16%|█▌ | 399MB / 2.49GB  ...nts/optim_065000_rank6.pt: 16%|█▋ | 405MB / 2.49GB  ...nts/optim_065000_rank7.pt: 16%|█▌ | 400MB / 2.49GB  ...eckpoints/model_065000.pt: 7%|▋ | 175MB / 2.35GB  Processing Files (0 / 8) : 13%|█▎ | 2.99GB / 22.2GB, 283MB/s New Data Upload : 47%|████▋ | 937MB / 2.01GB, 88.8MB/s  ...nts/optim_065000_rank0.pt: 17%|█▋ | 421MB / 2.49GB  ...nts/optim_065000_rank1.pt: 17%|█▋ | 425MB / 2.49GB  ...nts/optim_065000_rank2.pt: 17%|█▋ | 416MB / 2.49GB  ...nts/optim_065000_rank4.pt: 17%|█▋ | 422MB / 2.49GB  ...nts/optim_065000_rank5.pt: 17%|█▋ | 418MB / 2.49GB  ...nts/optim_065000_rank6.pt: 17%|█▋ | 424MB / 2.49GB  ...nts/optim_065000_rank7.pt: 17%|█▋ | 420MB / 2.49GB  ...eckpoints/model_065000.pt: 8%|▊ | 176MB / 2.35GB  Processing Files (0 / 8) : 14%|█▍ | 3.12GB / 22.2GB, 295MB/s New Data Upload : 48%|████▊ | 964MB / 2.01GB, 91.1MB/s  ...nts/optim_065000_rank0.pt: 18%|█▊ | 441MB / 2.49GB  ...nts/optim_065000_rank1.pt: 18%|█▊ | 441MB / 2.49GB  ...nts/optim_065000_rank2.pt: 17%|█▋ | 434MB / 2.49GB  ...nts/optim_065000_rank4.pt: 18%|█▊ | 439MB / 2.49GB  ...nts/optim_065000_rank5.pt: 17%|█▋ | 435MB / 2.49GB  ...nts/optim_065000_rank6.pt: 18%|█▊ | 440MB / 2.49GB  ...nts/optim_065000_rank7.pt: 18%|█▊ | 436MB / 2.49GB  ...eckpoints/model_065000.pt: 8%|▊ | 185MB / 2.35GB  Processing Files (0 / 8) : 15%|█▍ | 3.25GB / 22.2GB, 306MB/s New Data Upload : 50%|████▉ | 999MB / 2.01GB, 94.2MB/s  ...nts/optim_065000_rank0.pt: 19%|█▊ | 463MB / 2.49GB  ...nts/optim_065000_rank1.pt: 19%|█▊ | 463MB / 2.49GB  ...nts/optim_065000_rank2.pt: 18%|█▊ | 454MB / 2.49GB  ...nts/optim_065000_rank4.pt: 19%|█▊ | 460MB / 2.49GB  ...nts/optim_065000_rank5.pt: 18%|█▊ | 456MB / 2.49GB  ...nts/optim_065000_rank6.pt: 19%|█▊ | 462MB / 2.49GB  ...nts/optim_065000_rank7.pt: 18%|█▊ | 458MB / 2.49GB  ...eckpoints/model_065000.pt: 8%|▊ | 186MB / 2.35GB  Processing Files (0 / 8) : 15%|█▌ | 3.40GB / 22.2GB, 319MB/s New Data Upload : 50%|████▉ | 1.03GB / 2.08GB, 97.0MB/s  ...nts/optim_065000_rank0.pt: 20%|█▉ | 497MB / 2.49GB  ...nts/optim_065000_rank1.pt: 20%|█▉ | 489MB / 2.49GB  ...nts/optim_065000_rank2.pt: 19%|█▉ | 480MB / 2.49GB  ...nts/optim_065000_rank4.pt: 20%|█▉ | 485MB / 2.49GB  ...nts/optim_065000_rank5.pt: 19%|█▉ | 480MB / 2.49GB  ...nts/optim_065000_rank6.pt: 20%|█▉ | 488MB / 2.49GB  ...nts/optim_065000_rank7.pt: 19%|█▉ | 484MB / 2.49GB  ...eckpoints/model_065000.pt: 8%|▊ | 196MB / 2.35GB  Processing Files (0 / 8) : 16%|█▌ | 3.60GB / 22.2GB, 338MB/s New Data Upload : 52%|█████▏ | 1.08GB / 2.08GB, 102MB/s  ...nts/optim_065000_rank0.pt: 21%|██ | 523MB / 2.49GB  ...nts/optim_065000_rank1.pt: 21%|██ | 511MB / 2.49GB  ...nts/optim_065000_rank2.pt: 20%|██ | 502MB / 2.49GB  ...nts/optim_065000_rank4.pt: 20%|██ | 507MB / 2.49GB  ...nts/optim_065000_rank5.pt: 20%|██ | 503MB / 2.49GB  ...nts/optim_065000_rank6.pt: 21%|██ | 511MB / 2.49GB  ...nts/optim_065000_rank7.pt: 20%|██ | 506MB / 2.49GB  ...eckpoints/model_065000.pt: 9%|▊ | 202MB / 2.35GB  Processing Files (0 / 8) : 17%|█▋ | 3.76GB / 22.2GB, 353MB/s New Data Upload : 51%|█████ | 1.12GB / 2.21GB, 105MB/s  ...nts/optim_065000_rank0.pt: 22%|██▏ | 546MB / 2.49GB  ...nts/optim_065000_rank1.pt: 22%|██▏ | 535MB / 2.49GB  ...nts/optim_065000_rank2.pt: 21%|██ | 522MB / 2.49GB  ...nts/optim_065000_rank4.pt: 21%|██▏ | 532MB / 2.49GB  ...nts/optim_065000_rank5.pt: 21%|██ | 527MB / 2.49GB  ...nts/optim_065000_rank6.pt: 21%|██▏ | 534MB / 2.49GB  ...nts/optim_065000_rank7.pt: 21%|██▏ | 530MB / 2.49GB  ...eckpoints/model_065000.pt: 9%|▊ | 203MB / 2.35GB  Processing Files (0 / 8) : 18%|█▊ | 3.93GB / 22.2GB, 367MB/s New Data Upload : 51%|█████ | 1.15GB / 2.28GB, 108MB/s  ...nts/optim_065000_rank0.pt: 23%|██▎ | 578MB / 2.49GB  ...nts/optim_065000_rank1.pt: 23%|██▎ | 565MB / 2.49GB  ...nts/optim_065000_rank2.pt: 22%|██▏ | 549MB / 2.49GB  ...nts/optim_065000_rank4.pt: 23%|██▎ | 562MB / 2.49GB  ...nts/optim_065000_rank5.pt: 22%|██▏ | 557MB / 2.49GB  ...nts/optim_065000_rank6.pt: 23%|██▎ | 564MB / 2.49GB  ...nts/optim_065000_rank7.pt: 23%|██▎ | 560MB / 2.49GB  ...eckpoints/model_065000.pt: 9%|▉ | 211MB / 2.35GB  Processing Files (0 / 8) : 19%|█▊ | 4.14GB / 22.2GB, 387MB/s New Data Upload : 50%|████▉ | 1.20GB / 2.41GB, 112MB/s  ...nts/optim_065000_rank0.pt: 24%|██▍ | 608MB / 2.49GB  ...nts/optim_065000_rank1.pt: 24%|██▍ | 592MB / 2.49GB  ...nts/optim_065000_rank2.pt: 23%|██▎ | 572MB / 2.49GB  ...nts/optim_065000_rank4.pt: 24%|██▎ | 589MB / 2.49GB  ...nts/optim_065000_rank5.pt: 24%|██▎ | 584MB / 2.49GB  ...nts/optim_065000_rank6.pt: 24%|██▍ | 591MB / 2.49GB  ...nts/optim_065000_rank7.pt: 24%|██▎ | 587MB / 2.49GB  ...eckpoints/model_065000.pt: 9%|▉ | 218MB / 2.35GB  Processing Files (0 / 8) : 20%|█▉ | 4.34GB / 22.2GB, 405MB/s New Data Upload : 52%|█████▏ | 1.25GB / 2.41GB, 116MB/s  ...nts/optim_065000_rank0.pt: 26%|██▌ | 635MB / 2.49GB  ...nts/optim_065000_rank1.pt: 25%|██▍ | 618MB / 2.49GB  ...nts/optim_065000_rank2.pt: 24%|██▍ | 594MB / 2.49GB  ...nts/optim_065000_rank4.pt: 25%|██▍ | 615MB / 2.49GB  ...nts/optim_065000_rank5.pt: 25%|██▍ | 610MB / 2.49GB  ...nts/optim_065000_rank6.pt: 25%|██▍ | 617MB / 2.49GB  ...nts/optim_065000_rank7.pt: 25%|██▍ | 613MB / 2.49GB  ...eckpoints/model_065000.pt: 10%|▉ | 224MB / 2.35GB  Processing Files (0 / 8) : 20%|██ | 4.53GB / 22.2GB, 422MB/s New Data Upload : 54%|█████▎ | 1.29GB / 2.41GB, 120MB/s  ...nts/optim_065000_rank0.pt: 27%|██▋ | 659MB / 2.49GB  ...nts/optim_065000_rank1.pt: 26%|██▌ | 641MB / 2.49GB  ...nts/optim_065000_rank2.pt: 25%|██▍ | 616MB / 2.49GB  ...nts/optim_065000_rank4.pt: 26%|██▌ | 641MB / 2.49GB  ...nts/optim_065000_rank5.pt: 26%|██▌ | 636MB / 2.49GB  ...nts/optim_065000_rank6.pt: 26%|██▌ | 640MB / 2.49GB  ...nts/optim_065000_rank7.pt: 26%|██▌ | 637MB / 2.49GB  ...eckpoints/model_065000.pt: 10%|▉ | 231MB / 2.35GB  Processing Files (0 / 8) : 21%|██ | 4.70GB / 22.2GB, 438MB/s New Data Upload : 54%|█████▎ | 1.33GB / 2.48GB, 124MB/s  ...nts/optim_065000_rank0.pt: 27%|██▋ | 683MB / 2.49GB  ...nts/optim_065000_rank1.pt: 27%|██▋ | 661MB / 2.49GB  ...nts/optim_065000_rank2.pt: 26%|██▌ | 635MB / 2.49GB  ...nts/optim_065000_rank4.pt: 27%|██▋ | 661MB / 2.49GB  ...nts/optim_065000_rank5.pt: 26%|██▋ | 656MB / 2.49GB  ...nts/optim_065000_rank6.pt: 27%|██▋ | 661MB / 2.49GB  ...nts/optim_065000_rank7.pt: 26%|██▋ | 658MB / 2.49GB  ...eckpoints/model_065000.pt: 10%|█ | 243MB / 2.35GB  Processing Files (0 / 8) : 22%|██▏ | 4.86GB / 22.2GB, 454MB/s New Data Upload : 55%|█████▌ | 1.37GB / 2.48GB, 128MB/s  ...nts/optim_065000_rank0.pt: 28%|██▊ | 702MB / 2.49GB  ...nts/optim_065000_rank1.pt: 27%|██▋ | 679MB / 2.49GB  ...nts/optim_065000_rank2.pt: 26%|██▌ | 651MB / 2.49GB  ...nts/optim_065000_rank4.pt: 27%|██▋ | 679MB / 2.49GB  ...nts/optim_065000_rank5.pt: 27%|██▋ | 674MB / 2.49GB  ...nts/optim_065000_rank6.pt: 27%|██▋ | 679MB / 2.49GB  ...nts/optim_065000_rank7.pt: 27%|██▋ | 676MB / 2.49GB  ...eckpoints/model_065000.pt: 11%|█ | 256MB / 2.35GB  Processing Files (0 / 8) : 22%|██▏ | 5.00GB / 22.2GB, 467MB/s New Data Upload : 57%|█████▋ | 1.41GB / 2.48GB, 132MB/s  ...nts/optim_065000_rank0.pt: 29%|██▉ | 721MB / 2.49GB  ...nts/optim_065000_rank1.pt: 28%|██▊ | 697MB / 2.49GB  ...nts/optim_065000_rank2.pt: 27%|██▋ | 671MB / 2.49GB  ...nts/optim_065000_rank4.pt: 28%|██▊ | 699MB / 2.49GB  ...nts/optim_065000_rank5.pt: 28%|██▊ | 694MB / 2.49GB  ...nts/optim_065000_rank6.pt: 28%|██▊ | 696MB / 2.49GB  ...nts/optim_065000_rank7.pt: 28%|██▊ | 693MB / 2.49GB  ...eckpoints/model_065000.pt: 11%|█▏ | 265MB / 2.35GB  Processing Files (0 / 8) : 23%|██▎ | 5.13GB / 22.2GB, 481MB/s New Data Upload : 57%|█████▋ | 1.45GB / 2.55GB, 136MB/s  ...nts/optim_065000_rank0.pt: 30%|███ | 748MB / 2.49GB  ...nts/optim_065000_rank1.pt: 29%|██▉ | 724MB / 2.49GB  ...nts/optim_065000_rank2.pt: 28%|██▊ | 696MB / 2.49GB  ...nts/optim_065000_rank4.pt: 29%|██▉ | 725MB / 2.49GB  ...nts/optim_065000_rank5.pt: 29%|██▉ | 720MB / 2.49GB  ...nts/optim_065000_rank6.pt: 29%|██▉ | 723MB / 2.49GB  ...nts/optim_065000_rank7.pt: 29%|██▉ | 720MB / 2.49GB  ...eckpoints/model_065000.pt: 12%|█▏ | 277MB / 2.35GB  Processing Files (0 / 8) : 24%|██▍ | 5.33GB / 22.2GB, 500MB/s New Data Upload : 59%|█████▉ | 1.50GB / 2.55GB, 140MB/s  ...nts/optim_065000_rank0.pt: 31%|███ | 776MB / 2.49GB  ...nts/optim_065000_rank1.pt: 30%|███ | 749MB / 2.49GB  ...nts/optim_065000_rank2.pt: 29%|██▉ | 722MB / 2.49GB  ...nts/optim_065000_rank4.pt: 30%|███ | 751MB / 2.49GB  ...nts/optim_065000_rank5.pt: 30%|███ | 746MB / 2.49GB  ...nts/optim_065000_rank6.pt: 30%|███ | 748MB / 2.49GB  ...nts/optim_065000_rank7.pt: 30%|██▉ | 745MB / 2.49GB  ...eckpoints/model_065000.pt: 12%|█▏ | 288MB / 2.35GB  Processing Files (0 / 8) : 25%|██▍ | 5.53GB / 22.2GB, 518MB/s New Data Upload : 59%|█████▉ | 1.55GB / 2.62GB, 145MB/s  ...nts/optim_065000_rank0.pt: 32%|███▏ | 792MB / 2.49GB  ...nts/optim_065000_rank1.pt: 31%|███ | 766MB / 2.49GB  ...nts/optim_065000_rank2.pt: 30%|██▉ | 740MB / 2.49GB  ...nts/optim_065000_rank4.pt: 31%|███ | 769MB / 2.49GB  ...nts/optim_065000_rank5.pt: 31%|███ | 763MB / 2.49GB  ...nts/optim_065000_rank6.pt: 31%|███ | 765MB / 2.49GB  ...nts/optim_065000_rank7.pt: 31%|███ | 762MB / 2.49GB  ...eckpoints/model_065000.pt: 13%|█▎ | 299MB / 2.35GB  Processing Files (0 / 8) : 25%|██▌ | 5.65GB / 22.2GB, 530MB/s New Data Upload : 59%|█████▉ | 1.58GB / 2.68GB, 148MB/s  ...nts/optim_065000_rank0.pt: 33%|███▎ | 815MB / 2.49GB  ...nts/optim_065000_rank1.pt: 32%|███▏ | 793MB / 2.49GB  ...nts/optim_065000_rank2.pt: 31%|███ | 767MB / 2.49GB  ...nts/optim_065000_rank4.pt: 32%|███▏ | 796MB / 2.49GB  ...nts/optim_065000_rank5.pt: 32%|███▏ | 790MB / 2.49GB  ...nts/optim_065000_rank6.pt: 32%|███▏ | 791MB / 2.49GB  ...nts/optim_065000_rank7.pt: 32%|███▏ | 788MB / 2.49GB  ...eckpoints/model_065000.pt: 13%|█▎ | 307MB / 2.35GB  Processing Files (0 / 8) : 26%|██▋ | 5.85GB / 22.2GB, 548MB/s New Data Upload : 61%|██████ | 1.62GB / 2.68GB, 151MB/s  ...nts/optim_065000_rank0.pt: 34%|███▍ | 839MB / 2.49GB  ...nts/optim_065000_rank1.pt: 33%|███▎ | 820MB / 2.49GB  ...nts/optim_065000_rank2.pt: 32%|███▏ | 796MB / 2.49GB  ...nts/optim_065000_rank4.pt: 33%|███▎ | 825MB / 2.49GB  ...nts/optim_065000_rank5.pt: 33%|███▎ | 817MB / 2.49GB  ...nts/optim_065000_rank6.pt: 33%|███▎ | 818MB / 2.49GB  ...nts/optim_065000_rank7.pt: 33%|███▎ | 815MB / 2.49GB  ...eckpoints/model_065000.pt: 14%|█▎ | 318MB / 2.35GB  Processing Files (0 / 8) : 27%|██▋ | 6.05GB / 22.2GB, 567MB/s New Data Upload : 61%|██████ | 1.67GB / 2.75GB, 155MB/s  ...nts/optim_065000_rank0.pt: 34%|███▍ | 857MB / 2.49GB  ...nts/optim_065000_rank1.pt: 34%|███▍ | 839MB / 2.49GB  ...nts/optim_065000_rank2.pt: 33%|███▎ | 815MB / 2.49GB  ...nts/optim_065000_rank4.pt: 34%|███▍ | 844MB / 2.49GB  ...nts/optim_065000_rank5.pt: 34%|███▎ | 837MB / 2.49GB  ...nts/optim_065000_rank6.pt: 34%|███▎ | 837MB / 2.49GB  ...nts/optim_065000_rank7.pt: 34%|███▎ | 834MB / 2.49GB  ...eckpoints/model_065000.pt: 14%|█▍ | 326MB / 2.35GB  Processing Files (0 / 8) : 28%|██▊ | 6.19GB / 22.2GB, 579MB/s New Data Upload : 62%|██████▏ | 1.70GB / 2.75GB, 157MB/s  ...nts/optim_065000_rank0.pt: 35%|███▌ | 872MB / 2.49GB  ...nts/optim_065000_rank1.pt: 35%|███▍ | 859MB / 2.49GB  ...nts/optim_065000_rank2.pt: 34%|███▎ | 834MB / 2.49GB  ...nts/optim_065000_rank4.pt: 35%|███▍ | 864MB / 2.49GB  ...nts/optim_065000_rank5.pt: 34%|███▍ | 857MB / 2.49GB  ...nts/optim_065000_rank6.pt: 34%|███▍ | 856MB / 2.49GB  ...nts/optim_065000_rank7.pt: 34%|███▍ | 852MB / 2.49GB  ...eckpoints/model_065000.pt: 14%|█▍ | 327MB / 2.35GB  Processing Files (0 / 8) : 28%|██▊ | 6.32GB / 22.2GB, 590MB/s New Data Upload : 62%|██████▏ | 1.73GB / 2.82GB, 159MB/s  ...nts/optim_065000_rank0.pt: 36%|███▌ | 900MB / 2.49GB  ...nts/optim_065000_rank1.pt: 36%|███▌ | 887MB / 2.49GB  ...nts/optim_065000_rank2.pt: 35%|███▍ | 863MB / 2.49GB  ...nts/optim_065000_rank4.pt: 36%|███▌ | 891MB / 2.49GB  ...nts/optim_065000_rank5.pt: 36%|███▌ | 885MB / 2.49GB  ...nts/optim_065000_rank6.pt: 36%|███▌ | 885MB / 2.49GB  ...nts/optim_065000_rank7.pt: 35%|███▌ | 881MB / 2.49GB  ...eckpoints/model_065000.pt: 14%|█▍ | 330MB / 2.35GB  Processing Files (0 / 8) : 29%|██▉ | 6.52GB / 22.2GB, 608MB/s New Data Upload : 60%|██████ | 1.77GB / 2.95GB, 161MB/s  ...nts/optim_065000_rank0.pt: 37%|███▋ | 923MB / 2.49GB  ...nts/optim_065000_rank1.pt: 37%|███▋ | 912MB / 2.49GB  ...nts/optim_065000_rank2.pt: 36%|███▌ | 890MB / 2.49GB  ...nts/optim_065000_rank4.pt: 37%|███▋ | 916MB / 2.49GB  ...nts/optim_065000_rank5.pt: 37%|███▋ | 912MB / 2.49GB  ...nts/optim_065000_rank6.pt: 37%|███▋ | 909MB / 2.49GB  ...nts/optim_065000_rank7.pt: 36%|███▋ | 905MB / 2.49GB  ...eckpoints/model_065000.pt: 14%|█▍ | 333MB / 2.35GB  Processing Files (0 / 8) : 30%|███ | 6.70GB / 22.2GB, 622MB/s New Data Upload : 59%|█████▉ | 1.81GB / 3.08GB, 163MB/s  ...nts/optim_065000_rank0.pt: 38%|███▊ | 947MB / 2.49GB  ...nts/optim_065000_rank1.pt: 38%|███▊ | 935MB / 2.49GB  ...nts/optim_065000_rank2.pt: 37%|███▋ | 915MB / 2.49GB  ...nts/optim_065000_rank4.pt: 38%|███▊ | 940MB / 2.49GB  ...nts/optim_065000_rank5.pt: 38%|███▊ | 937MB / 2.49GB  ...nts/optim_065000_rank6.pt: 38%|███▊ | 933MB / 2.49GB  ...nts/optim_065000_rank7.pt: 37%|███▋ | 929MB / 2.49GB  ...eckpoints/model_065000.pt: 14%|█▍ | 333MB / 2.35GB  Processing Files (0 / 8) : 31%|███ | 6.87GB / 22.2GB, 636MB/s New Data Upload : 57%|█████▋ | 1.85GB / 3.22GB, 165MB/s  ...nts/optim_065000_rank0.pt: 39%|███▉ | 972MB / 2.49GB  ...nts/optim_065000_rank1.pt: 39%|███▉ | 967MB / 2.49GB  ...nts/optim_065000_rank2.pt: 38%|███▊ | 941MB / 2.49GB  ...nts/optim_065000_rank4.pt: 39%|███▉ | 966MB / 2.49GB  ...nts/optim_065000_rank5.pt: 39%|███▉ | 969MB / 2.49GB  ...nts/optim_065000_rank6.pt: 39%|███▊ | 961MB / 2.49GB  ...nts/optim_065000_rank7.pt: 38%|███▊ | 956MB / 2.49GB  ...eckpoints/model_065000.pt: 14%|█▍ | 336MB / 2.35GB  Processing Files (0 / 8) : 32%|███▏ | 7.07GB / 22.2GB, 653MB/s New Data Upload : 58%|█████▊ | 1.89GB / 3.29GB, 168MB/s  ...nts/optim_065000_rank0.pt: 40%|████ | 994MB / 2.49GB  ...nts/optim_065000_rank1.pt: 40%|███▉ | 992MB / 2.49GB  ...nts/optim_065000_rank2.pt: 39%|███▊ | 963MB / 2.49GB  ...nts/optim_065000_rank4.pt: 40%|███▉ | 991MB / 2.49GB  ...nts/optim_065000_rank5.pt: 40%|███▉ | 993MB / 2.49GB  ...nts/optim_065000_rank6.pt: 40%|███▉ | 985MB / 2.49GB  ...nts/optim_065000_rank7.pt: 39%|███▉ | 980MB / 2.49GB  ...eckpoints/model_065000.pt: 15%|█▍ | 342MB / 2.35GB  Processing Files (0 / 8) : 33%|███▎ | 7.24GB / 22.2GB, 667MB/s New Data Upload : 57%|█████▋ | 1.93GB / 3.42GB, 170MB/s  ...nts/optim_065000_rank0.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_065000_rank1.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_065000_rank2.pt: 40%|████ | 995MB / 2.49GB  ...nts/optim_065000_rank4.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_065000_rank5.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_065000_rank6.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_065000_rank7.pt: 41%|████ | 1.01GB / 2.49GB  ...eckpoints/model_065000.pt: 15%|█▍ | 346MB / 2.35GB  Processing Files (0 / 8) : 34%|███▎ | 7.45GB / 22.2GB, 684MB/s New Data Upload : 58%|█████▊ | 1.98GB / 3.42GB, 173MB/s  ...nts/optim_065000_rank0.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_065000_rank1.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_065000_rank2.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_065000_rank4.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_065000_rank5.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_065000_rank6.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_065000_rank7.pt: 42%|████▏ | 1.03GB / 2.49GB  ...eckpoints/model_065000.pt: 15%|█▍ | 349MB / 2.35GB  Processing Files (0 / 8) : 34%|███▍ | 7.65GB / 22.2GB, 699MB/s New Data Upload : 57%|█████▋ | 2.03GB / 3.55GB, 175MB/s  ...nts/optim_065000_rank0.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_065000_rank1.pt: 43%|████▎ | 1.08GB / 2.49GB  ...nts/optim_065000_rank2.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_065000_rank4.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_065000_rank5.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_065000_rank6.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_065000_rank7.pt: 43%|████▎ | 1.06GB / 2.49GB  ...eckpoints/model_065000.pt: 15%|█▌ | 357MB / 2.35GB  Processing Files (0 / 8) : 35%|███▌ | 7.82GB / 22.2GB, 713MB/s New Data Upload : 58%|█████▊ | 2.07GB / 3.55GB, 177MB/s  ...nts/optim_065000_rank0.pt: 44%|████▍ | 1.09GB / 2.49GB  ...nts/optim_065000_rank1.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_065000_rank2.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_065000_rank4.pt: 44%|████▍ | 1.09GB / 2.49GB  ...nts/optim_065000_rank5.pt: 44%|████▍ | 1.09GB / 2.49GB  ...nts/optim_065000_rank6.pt: 44%|████▍ | 1.09GB / 2.49GB  ...nts/optim_065000_rank7.pt: 43%|████▎ | 1.08GB / 2.49GB  ...eckpoints/model_065000.pt: 15%|█▌ | 363MB / 2.35GB  Processing Files (0 / 8) : 36%|███▌ | 7.99GB / 22.2GB, 725MB/s New Data Upload : 59%|█████▉ | 2.11GB / 3.55GB, 179MB/s  ...nts/optim_065000_rank0.pt: 45%|████▍ | 1.12GB / 2.49GB  ...nts/optim_065000_rank1.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_065000_rank2.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_065000_rank4.pt: 45%|████▍ | 1.12GB / 2.49GB  ...nts/optim_065000_rank5.pt: 45%|████▌ | 1.12GB / 2.49GB  ...nts/optim_065000_rank6.pt: 45%|████▌ | 1.12GB / 2.49GB  ...nts/optim_065000_rank7.pt: 45%|████▍ | 1.11GB / 2.49GB  ...eckpoints/model_065000.pt: 16%|█▌ | 366MB / 2.35GB  Processing Files (0 / 8) : 37%|███▋ | 8.18GB / 22.2GB, 741MB/s New Data Upload : 59%|█████▉ | 2.15GB / 3.62GB, 181MB/s  ...nts/optim_065000_rank0.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_065000_rank1.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_065000_rank2.pt: 45%|████▌ | 1.13GB / 2.49GB  ...nts/optim_065000_rank4.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_065000_rank5.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_065000_rank6.pt: 46%|████▋ | 1.15GB / 2.49GB  ...nts/optim_065000_rank7.pt: 46%|████▌ | 1.13GB / 2.49GB  ...eckpoints/model_065000.pt: 16%|█▌ | 373MB / 2.35GB  Processing Files (0 / 8) : 38%|███▊ | 8.38GB / 22.2GB, 755MB/s New Data Upload : 61%|██████ | 2.20GB / 3.62GB, 183MB/s  ...nts/optim_065000_rank0.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_065000_rank1.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_065000_rank2.pt: 46%|████▌ | 1.15GB / 2.49GB  ...nts/optim_065000_rank4.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_065000_rank5.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_065000_rank6.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_065000_rank7.pt: 46%|████▋ | 1.15GB / 2.49GB  ...eckpoints/model_065000.pt: 16%|█▋ | 387MB / 2.35GB  Processing Files (0 / 8) : 38%|███▊ | 8.54GB / 22.2GB, 764MB/s New Data Upload : 62%|██████▏ | 2.24GB / 3.62GB, 184MB/s  ...nts/optim_065000_rank0.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_065000_rank1.pt: 49%|████▉ | 1.21GB / 2.49GB  ...nts/optim_065000_rank2.pt: 47%|████▋ | 1.18GB / 2.49GB  ...nts/optim_065000_rank4.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_065000_rank5.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_065000_rank6.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_065000_rank7.pt: 48%|████▊ | 1.18GB / 2.49GB  ...eckpoints/model_065000.pt: 17%|█▋ | 396MB / 2.35GB  Processing Files (0 / 8) : 39%|███▉ | 8.74GB / 22.2GB, 780MB/s New Data Upload : 62%|██████▏ | 2.29GB / 3.69GB, 187MB/s  ...nts/optim_065000_rank0.pt: 49%|████▊ | 1.21GB / 2.49GB  ...nts/optim_065000_rank1.pt: 49%|████▉ | 1.23GB / 2.49GB  ...nts/optim_065000_rank2.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_065000_rank4.pt: 49%|████▊ | 1.21GB / 2.49GB  ...nts/optim_065000_rank5.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_065000_rank6.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_065000_rank7.pt: 48%|████▊ | 1.20GB / 2.49GB  ...eckpoints/model_065000.pt: 17%|█▋ | 404MB / 2.35GB  Processing Files (0 / 8) : 40%|███▉ | 8.86GB / 22.2GB, 783MB/s New Data Upload : 63%|██████▎ | 2.33GB / 3.69GB, 186MB/s  ...nts/optim_065000_rank0.pt: 50%|████▉ | 1.23GB / 2.49GB  ...nts/optim_065000_rank1.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_065000_rank2.pt: 49%|████▉ | 1.21GB / 2.49GB  ...nts/optim_065000_rank4.pt: 49%|████▉ | 1.23GB / 2.49GB  ...nts/optim_065000_rank5.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_065000_rank6.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_065000_rank7.pt: 49%|████▉ | 1.22GB / 2.49GB  ...eckpoints/model_065000.pt: 18%|█▊ | 416MB / 2.35GB  Processing Files (0 / 8) : 41%|████ | 9.02GB / 22.2GB, 794MB/s New Data Upload : 64%|██████▍ | 2.37GB / 3.69GB, 189MB/s  ...nts/optim_065000_rank0.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_065000_rank1.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_065000_rank2.pt: 50%|████▉ | 1.23GB / 2.49GB  ...nts/optim_065000_rank4.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_065000_rank5.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_065000_rank6.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_065000_rank7.pt: 50%|████▉ | 1.24GB / 2.49GB  ...eckpoints/model_065000.pt: 18%|█▊ | 426MB / 2.35GB  Processing Files (0 / 8) : 41%|████ | 9.16GB / 22.2GB, 803MB/s New Data Upload : 64%|██████▍ | 2.42GB / 3.76GB, 191MB/s  ...nts/optim_065000_rank0.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_065000_rank1.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_065000_rank2.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_065000_rank4.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_065000_rank5.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_065000_rank6.pt: 52%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_065000_rank7.pt: 51%|█████ | 1.26GB / 2.49GB  ...eckpoints/model_065000.pt: 18%|█▊ | 432MB / 2.35GB  Processing Files (0 / 8) : 42%|████▏ | 9.33GB / 22.2GB, 816MB/s New Data Upload : 64%|██████▍ | 2.46GB / 3.82GB, 193MB/s  ...nts/optim_065000_rank0.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_065000_rank1.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_065000_rank2.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_065000_rank4.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_065000_rank5.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_065000_rank6.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_065000_rank7.pt: 52%|█████▏ | 1.28GB / 2.49GB  ...eckpoints/model_065000.pt: 19%|█▉ | 441MB / 2.35GB  Processing Files (0 / 8) : 43%|████▎ | 9.48GB / 22.2GB, 826MB/s New Data Upload : 63%|██████▎ | 2.51GB / 3.96GB, 196MB/s  ...nts/optim_065000_rank0.pt: 53%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_065000_rank1.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_065000_rank2.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_065000_rank4.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_065000_rank5.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_065000_rank6.pt: 53%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_065000_rank7.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...eckpoints/model_065000.pt: 19%|█▉ | 445MB / 2.35GB  Processing Files (0 / 8) : 44%|████▎ | 9.72GB / 22.2GB, 839MB/s New Data Upload : 64%|██████▎ | 2.56GB / 4.02GB, 198MB/s  ...nts/optim_065000_rank0.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_065000_rank1.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_065000_rank2.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_065000_rank4.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_065000_rank5.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_065000_rank6.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_065000_rank7.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...eckpoints/model_065000.pt: 19%|█▉ | 456MB / 2.35GB  Processing Files (0 / 8) : 45%|████▍ | 9.92GB / 22.2GB, 853MB/s New Data Upload : 65%|██████▍ | 2.61GB / 4.02GB, 201MB/s  ...nts/optim_065000_rank0.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_065000_rank1.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...nts/optim_065000_rank2.pt: 55%|█████▍ | 1.37GB / 2.49GB  ...nts/optim_065000_rank4.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_065000_rank5.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_065000_rank6.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_065000_rank7.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...eckpoints/model_065000.pt: 19%|█▉ | 456MB / 2.35GB  Processing Files (0 / 8) : 45%|████▌ | 10.1GB / 22.2GB, 854MB/s New Data Upload : 65%|██████▍ | 2.65GB / 4.09GB, 202MB/s  ...nts/optim_065000_rank0.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...nts/optim_065000_rank1.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_065000_rank2.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_065000_rank4.pt: 56%|█████▌ | 1.40GB / 2.49GB  ...nts/optim_065000_rank5.pt: 56%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_065000_rank6.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_065000_rank7.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...eckpoints/model_065000.pt: 20%|█▉ | 466MB / 2.35GB  Processing Files (0 / 8) : 46%|████▌ | 10.3GB / 22.2GB, 855MB/s New Data Upload : 66%|██████▌ | 2.70GB / 4.09GB, 203MB/s  ...nts/optim_065000_rank0.pt: 58%|█████▊ | 1.43GB / 2.49GB  ...nts/optim_065000_rank1.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_065000_rank2.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_065000_rank4.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_065000_rank5.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...nts/optim_065000_rank6.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_065000_rank7.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...eckpoints/model_065000.pt: 20%|█▉ | 468MB / 2.35GB  Processing Files (0 / 8) : 47%|████▋ | 10.5GB / 22.2GB, 862MB/s New Data Upload : 67%|██████▋ | 2.75GB / 4.09GB, 204MB/s  ...nts/optim_065000_rank0.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_065000_rank1.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_065000_rank4.pt: 59%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_065000_rank5.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_065000_rank6.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_065000_rank7.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...eckpoints/model_065000.pt: 20%|██ | 471MB / 2.35GB  Processing Files (0 / 8) : 48%|████▊ | 10.7GB / 22.2GB, 870MB/s New Data Upload : 66%|██████▌ | 2.79GB / 4.22GB, 206MB/s  ...nts/optim_065000_rank0.pt: 59%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_065000_rank1.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_065000_rank2.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_065000_rank4.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_065000_rank5.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_065000_rank6.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_065000_rank7.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...eckpoints/model_065000.pt: 20%|██ | 473MB / 2.35GB  Processing Files (0 / 8) : 49%|████▊ | 10.8GB / 22.2GB, 866MB/s New Data Upload : 65%|██████▍ | 2.82GB / 4.36GB, 205MB/s  ...nts/optim_065000_rank0.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_065000_rank1.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_065000_rank2.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_065000_rank4.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_065000_rank5.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_065000_rank6.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 60%|██████ | 1.50GB / 2.49GB  ...eckpoints/model_065000.pt: 20%|██ | 477MB / 2.35GB  Processing Files (0 / 8) : 49%|████▉ | 11.0GB / 22.2GB, 867MB/s New Data Upload : 65%|██████▌ | 2.85GB / 4.36GB, 205MB/s  ...nts/optim_065000_rank0.pt: 61%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_065000_rank1.pt: 63%|██████▎ | 1.55GB / 2.49GB  ...nts/optim_065000_rank2.pt: 61%|██████ | 1.52GB / 2.49GB  ...nts/optim_065000_rank4.pt: 61%|██████ | 1.52GB / 2.49GB  ...nts/optim_065000_rank5.pt: 61%|██████ | 1.50GB / 2.49GB  ...nts/optim_065000_rank6.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_065000_rank7.pt: 61%|██████ | 1.52GB / 2.49GB  ...eckpoints/model_065000.pt: 20%|██ | 481MB / 2.35GB  Processing Files (0 / 8) : 50%|█████ | 11.1GB / 22.2GB, 867MB/s New Data Upload : 66%|██████▋ | 2.89GB / 4.36GB, 206MB/s  ...nts/optim_065000_rank0.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_065000_rank1.pt: 64%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_065000_rank2.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_065000_rank4.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_065000_rank5.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_065000_rank6.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_065000_rank7.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...eckpoints/model_065000.pt: 21%|██ | 484MB / 2.35GB  Processing Files (0 / 8) : 51%|█████ | 11.4GB / 22.2GB, 871MB/s New Data Upload : 66%|██████▋ | 2.94GB / 4.43GB, 207MB/s  ...nts/optim_065000_rank0.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_065000_rank1.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_065000_rank2.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_065000_rank4.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_065000_rank5.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_065000_rank6.pt: 63%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_065000_rank7.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...eckpoints/model_065000.pt: 21%|██ | 490MB / 2.35GB  Processing Files (0 / 8) : 52%|█████▏ | 11.6GB / 22.2GB, 877MB/s New Data Upload : 68%|██████▊ | 2.99GB / 4.43GB, 208MB/s  ...nts/optim_065000_rank0.pt: 65%|██████▍ | 1.62GB / 2.49GB  ...nts/optim_065000_rank1.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_065000_rank2.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_065000_rank4.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_065000_rank5.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_065000_rank6.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_065000_rank7.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...eckpoints/model_065000.pt: 21%|██ | 495MB / 2.35GB  Processing Files (0 / 8) : 53%|█████▎ | 11.8GB / 22.2GB, 875MB/s New Data Upload : 68%|██████▊ | 3.03GB / 4.49GB, 209MB/s  ...nts/optim_065000_rank0.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_065000_rank1.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_065000_rank2.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_065000_rank4.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_065000_rank5.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_065000_rank6.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_065000_rank7.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...eckpoints/model_065000.pt: 21%|██▏ | 502MB / 2.35GB  Processing Files (0 / 8) : 54%|█████▍ | 12.0GB / 22.2GB, 884MB/s New Data Upload : 65%|██████▍ | 3.09GB / 4.76GB, 211MB/s  ...nts/optim_065000_rank3.pt: 3%|▎ | 63.9MB / 2.49GB  ...nts/optim_065000_rank0.pt: 67%|██████▋ | 1.68GB / 2.49GB  ...nts/optim_065000_rank1.pt: 69%|██████▊ | 1.71GB / 2.49GB  ...nts/optim_065000_rank2.pt: 67%|██████▋ | 1.68GB / 2.49GB  ...nts/optim_065000_rank4.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_065000_rank5.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_065000_rank6.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_065000_rank7.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...eckpoints/model_065000.pt: 22%|██▏ | 507MB / 2.35GB  ...nts/optim_065000_rank3.pt: 3%|▎ | 63.9MB / 2.49GB  Processing Files (0 / 9) : 55%|█████▌ | 12.3GB / 22.2GB, 898MB/s New Data Upload : 66%|██████▌ | 3.13GB / 4.76GB, 213MB/s  ...nts/optim_065000_rank0.pt: 69%|██████▊ | 1.71GB / 2.49GB  ...nts/optim_065000_rank1.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_065000_rank2.pt: 69%|██████▊ | 1.71GB / 2.49GB  ...nts/optim_065000_rank4.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_065000_rank5.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_065000_rank6.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_065000_rank7.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...eckpoints/model_065000.pt: 22%|██▏ | 515MB / 2.35GB  ...nts/optim_065000_rank3.pt: 8%|▊ | 203MB / 2.49GB  Processing Files (0 / 9) : 57%|█████▋ | 12.7GB / 22.2GB, 922MB/s New Data Upload : 67%|██████▋ | 3.19GB / 4.76GB, 215MB/s  ...nts/optim_065000_rank0.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_065000_rank1.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_065000_rank2.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_065000_rank4.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_065000_rank5.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_065000_rank6.pt: 69%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_065000_rank7.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...eckpoints/model_065000.pt: 22%|██▏ | 524MB / 2.35GB  ...nts/optim_065000_rank3.pt: 14%|█▎ | 339MB / 2.49GB  Processing Files (0 / 9) : 59%|█████▊ | 13.0GB / 22.2GB, 943MB/s New Data Upload : 68%|██████▊ | 3.24GB / 4.76GB, 217MB/s  ...nts/optim_065000_rank0.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_065000_rank1.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_065000_rank2.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_065000_rank4.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_065000_rank5.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_065000_rank6.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_065000_rank7.pt: 71%|███████ | 1.76GB / 2.49GB  ...eckpoints/model_065000.pt: 23%|██▎ | 530MB / 2.35GB  ...nts/optim_065000_rank3.pt: 15%|█▍ | 362MB / 2.49GB  Processing Files (0 / 9) : 59%|█████▉ | 13.2GB / 22.2GB, 943MB/s New Data Upload : 69%|██████▉ | 3.29GB / 4.76GB, 216MB/s  ...nts/optim_065000_rank0.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_065000_rank1.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_065000_rank2.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_065000_rank4.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_065000_rank5.pt: 71%|███████▏ | 1.77GB / 2.49GB  ...nts/optim_065000_rank6.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_065000_rank7.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...eckpoints/model_065000.pt: 23%|██▎ | 535MB / 2.35GB  ...nts/optim_065000_rank3.pt: 22%|██▏ | 542MB / 2.49GB  Processing Files (0 / 9) : 61%|██████ | 13.6GB / 22.2GB, 964MB/s New Data Upload : 70%|██████▉ | 3.33GB / 4.76GB, 217MB/s  ...nts/optim_065000_rank0.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_065000_rank1.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_065000_rank2.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_065000_rank4.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_065000_rank5.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_065000_rank6.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_065000_rank7.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...eckpoints/model_065000.pt: 23%|██▎ | 542MB / 2.35GB  ...nts/optim_065000_rank3.pt: 25%|██▍ | 620MB / 2.49GB  Processing Files (0 / 9) : 62%|██████▏ | 13.9GB / 22.2GB, 977MB/s New Data Upload : 71%|███████ | 3.38GB / 4.76GB, 219MB/s  ...nts/optim_065000_rank0.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_065000_rank1.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_065000_rank2.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_065000_rank4.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_065000_rank5.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_065000_rank6.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_065000_rank7.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...eckpoints/model_065000.pt: 23%|██▎ | 547MB / 2.35GB  ...nts/optim_065000_rank3.pt: 30%|███ | 755MB / 2.49GB  Processing Files (0 / 9) : 64%|██████▍ | 14.2GB / 22.2GB, 983MB/s New Data Upload : 72%|███████▏ | 3.41GB / 4.76GB, 217MB/s  ...nts/optim_065000_rank0.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_065000_rank1.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_065000_rank2.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_065000_rank4.pt: 75%|███████▌ | 1.86GB / 2.49GB  ...nts/optim_065000_rank5.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_065000_rank6.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_065000_rank7.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...eckpoints/model_065000.pt: 23%|██▎ | 551MB / 2.35GB  ...nts/optim_065000_rank3.pt: 35%|███▍ | 862MB / 2.49GB  Processing Files (0 / 9) : 65%|██████▌ | 14.5GB / 22.2GB, 995MB/s New Data Upload : 72%|███████▏ | 3.45GB / 4.83GB, 216MB/s  ...nts/optim_065000_rank0.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_065000_rank1.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_065000_rank2.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_065000_rank4.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_065000_rank5.pt: 75%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_065000_rank6.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_065000_rank7.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...eckpoints/model_065000.pt: 24%|██▎ | 555MB / 2.35GB  ...nts/optim_065000_rank3.pt: 40%|████ | 999MB / 2.49GB  Processing Files (0 / 9) : 67%|██████▋ | 14.8GB / 22.2GB, 1.01GB/s New Data Upload : 72%|███████▏ | 3.50GB / 4.83GB, 216MB/s  ...nts/optim_065000_rank0.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_065000_rank1.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_065000_rank2.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_065000_rank4.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_065000_rank5.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_065000_rank6.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_065000_rank7.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...eckpoints/model_065000.pt: 24%|██▍ | 562MB / 2.35GB  ...nts/optim_065000_rank3.pt: 42%|████▏ | 1.04GB / 2.49GB  Processing Files (0 / 9) : 67%|██████▋ | 15.0GB / 22.2GB, 1.01GB/s New Data Upload : 73%|███████▎ | 3.53GB / 4.83GB, 216MB/s  ...nts/optim_065000_rank0.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_065000_rank1.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_065000_rank2.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_065000_rank4.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_065000_rank5.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_065000_rank6.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_065000_rank7.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...eckpoints/model_065000.pt: 24%|██▍ | 567MB / 2.35GB  ...nts/optim_065000_rank3.pt: 45%|████▌ | 1.13GB / 2.49GB  Processing Files (0 / 9) : 68%|██████▊ | 15.2GB / 22.2GB, 1.02GB/s New Data Upload : 74%|███████▍ | 3.58GB / 4.83GB, 216MB/s  ...nts/optim_065000_rank0.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_065000_rank1.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_065000_rank2.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_065000_rank4.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_065000_rank5.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_065000_rank6.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_065000_rank7.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...eckpoints/model_065000.pt: 24%|██▍ | 571MB / 2.35GB  ...nts/optim_065000_rank3.pt: 49%|████▉ | 1.23GB / 2.49GB  Processing Files (0 / 9) : 69%|██████▉ | 15.4GB / 22.2GB, 1.02GB/s New Data Upload : 74%|███████▎ | 3.61GB / 4.90GB, 215MB/s  ...nts/optim_065000_rank0.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_065000_rank1.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_065000_rank2.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_065000_rank4.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_065000_rank5.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_065000_rank6.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_065000_rank7.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...eckpoints/model_065000.pt: 25%|██▍ | 582MB / 2.35GB  ...nts/optim_065000_rank3.pt: 55%|█████▍ | 1.36GB / 2.49GB  Processing Files (0 / 9) : 71%|███████ | 15.7GB / 22.2GB, 1.04GB/s New Data Upload : 75%|███████▍ | 3.66GB / 4.90GB, 216MB/s  ...nts/optim_065000_rank0.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_065000_rank1.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_065000_rank2.pt: 80%|███████▉ | 1.99GB / 2.49GB  ...nts/optim_065000_rank4.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_065000_rank5.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_065000_rank6.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_065000_rank7.pt: 80%|████████ | 1.99GB / 2.49GB  ...eckpoints/model_065000.pt: 25%|██▌ | 587MB / 2.35GB  ...nts/optim_065000_rank3.pt: 61%|██████ | 1.52GB / 2.49GB  Processing Files (0 / 9) : 72%|███████▏ | 16.1GB / 22.2GB, 1.05GB/s New Data Upload : 75%|███████▍ | 3.70GB / 4.96GB, 216MB/s  ...nts/optim_065000_rank0.pt: 81%|████████ | 2.00GB / 2.49GB  ...nts/optim_065000_rank1.pt: 81%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_065000_rank2.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_065000_rank4.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_065000_rank5.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_065000_rank6.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_065000_rank7.pt: 81%|████████ | 2.01GB / 2.49GB  ...eckpoints/model_065000.pt: 25%|██▌ | 598MB / 2.35GB  ...nts/optim_065000_rank3.pt: 63%|██████▎ | 1.57GB / 2.49GB  Processing Files (0 / 9) : 73%|███████▎ | 16.2GB / 22.2GB, 1.05GB/s New Data Upload : 75%|███████▌ | 3.74GB / 4.96GB, 215MB/s  ...nts/optim_065000_rank0.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_065000_rank1.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_065000_rank2.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_065000_rank4.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_065000_rank5.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_065000_rank6.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_065000_rank7.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...eckpoints/model_065000.pt: 26%|██▌ | 606MB / 2.35GB  ...nts/optim_065000_rank3.pt: 67%|██████▋ | 1.66GB / 2.49GB  Processing Files (0 / 9) : 74%|███████▍ | 16.5GB / 22.2GB, 1.06GB/s New Data Upload : 75%|███████▌ | 3.80GB / 5.03GB, 217MB/s  ...nts/optim_065000_rank0.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_065000_rank1.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_065000_rank2.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_065000_rank4.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_065000_rank5.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_065000_rank6.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_065000_rank7.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...eckpoints/model_065000.pt: 26%|██▌ | 614MB / 2.35GB  ...nts/optim_065000_rank3.pt: 74%|███████▎ | 1.83GB / 2.49GB  Processing Files (0 / 9) : 76%|███████▌ | 16.8GB / 22.2GB, 1.07GB/s New Data Upload : 76%|███████▋ | 3.83GB / 5.03GB, 217MB/s  ...nts/optim_065000_rank0.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_065000_rank1.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_065000_rank2.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_065000_rank4.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_065000_rank5.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_065000_rank6.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_065000_rank7.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...eckpoints/model_065000.pt: 26%|██▋ | 620MB / 2.35GB  ...nts/optim_065000_rank3.pt: 76%|███████▌ | 1.88GB / 2.49GB  Processing Files (0 / 9) : 76%|███████▋ | 17.0GB / 22.2GB, 1.07GB/s New Data Upload : 77%|███████▋ | 3.88GB / 5.03GB, 216MB/s  ...nts/optim_065000_rank0.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_065000_rank1.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_065000_rank2.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_065000_rank4.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_065000_rank5.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_065000_rank6.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_065000_rank7.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...eckpoints/model_065000.pt: 27%|██▋ | 632MB / 2.35GB  ...nts/optim_065000_rank3.pt: 81%|████████ | 2.01GB / 2.49GB  Processing Files (0 / 9) : 77%|███████▋ | 17.2GB / 22.2GB, 1.08GB/s New Data Upload : 78%|███████▊ | 3.92GB / 5.03GB, 218MB/s  ...nts/optim_065000_rank0.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_065000_rank1.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_065000_rank2.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_065000_rank4.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_065000_rank5.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_065000_rank6.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_065000_rank7.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...eckpoints/model_065000.pt: 27%|██▋ | 640MB / 2.35GB  ...nts/optim_065000_rank3.pt: 83%|████████▎ | 2.07GB / 2.49GB  Processing Files (0 / 9) : 78%|███████▊ | 17.4GB / 22.2GB, 1.09GB/s New Data Upload : 78%|███████▊ | 3.96GB / 5.10GB, 218MB/s  ...nts/optim_065000_rank0.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_065000_rank1.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_065000_rank2.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_065000_rank4.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_065000_rank5.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_065000_rank6.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_065000_rank7.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...eckpoints/model_065000.pt: 28%|██▊ | 659MB / 2.35GB  ...nts/optim_065000_rank3.pt: 85%|████████▍ | 2.10GB / 2.49GB  Processing Files (0 / 9) : 79%|███████▉ | 17.6GB / 22.2GB, 1.08GB/s New Data Upload : 79%|███████▊ | 4.01GB / 5.10GB, 219MB/s  ...nts/optim_065000_rank0.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_065000_rank1.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_065000_rank2.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_065000_rank4.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_065000_rank5.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_065000_rank6.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_065000_rank7.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...eckpoints/model_065000.pt: 28%|██▊ | 666MB / 2.35GB  ...nts/optim_065000_rank3.pt: 85%|████████▌ | 2.12GB / 2.49GB  Processing Files (0 / 9) : 80%|███████▉ | 17.7GB / 22.2GB, 1.08GB/s New Data Upload : 78%|███████▊ | 4.05GB / 5.16GB, 219MB/s  ...nts/optim_065000_rank0.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_065000_rank1.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_065000_rank2.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_065000_rank4.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_065000_rank5.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_065000_rank6.pt: 86%|████████▋ | 2.14GB / 2.49GB  ...nts/optim_065000_rank7.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...eckpoints/model_065000.pt: 29%|██▉ | 687MB / 2.35GB  ...nts/optim_065000_rank3.pt: 86%|████████▌ | 2.14GB / 2.49GB  Processing Files (0 / 9) : 80%|████████ | 17.8GB / 22.2GB, 1.08GB/s New Data Upload : 78%|███████▊ | 4.09GB / 5.23GB, 220MB/s  ...nts/optim_065000_rank0.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_065000_rank1.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_065000_rank2.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_065000_rank4.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_065000_rank5.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_065000_rank6.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_065000_rank7.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...eckpoints/model_065000.pt: 30%|██▉ | 698MB / 2.35GB  ...nts/optim_065000_rank3.pt: 87%|████████▋ | 2.15GB / 2.49GB  Processing Files (0 / 9) : 81%|████████ | 18.0GB / 22.2GB, 1.07GB/s New Data Upload : 79%|███████▉ | 4.13GB / 5.23GB, 220MB/s  ...nts/optim_065000_rank0.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_065000_rank1.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_065000_rank2.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_065000_rank4.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_065000_rank5.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_065000_rank6.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_065000_rank7.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...eckpoints/model_065000.pt: 30%|███ | 716MB / 2.35GB  ...nts/optim_065000_rank3.pt: 87%|████████▋ | 2.17GB / 2.49GB  Processing Files (0 / 9) : 81%|████████▏ | 18.1GB / 22.2GB, 1.06GB/s New Data Upload : 80%|███████▉ | 4.17GB / 5.23GB, 219MB/s  ...nts/optim_065000_rank0.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_065000_rank1.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_065000_rank2.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_065000_rank4.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_065000_rank5.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_065000_rank6.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_065000_rank7.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...eckpoints/model_065000.pt: 31%|███ | 732MB / 2.35GB  ...nts/optim_065000_rank3.pt: 88%|████████▊ | 2.18GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.2GB / 22.2GB, 1.06GB/s New Data Upload : 80%|███████▉ | 4.22GB / 5.30GB, 219MB/s  ...nts/optim_065000_rank0.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_065000_rank1.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_065000_rank2.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_065000_rank4.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_065000_rank5.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_065000_rank6.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_065000_rank7.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...eckpoints/model_065000.pt: 32%|███▏ | 750MB / 2.35GB  ...nts/optim_065000_rank3.pt: 89%|████████▊ | 2.20GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.4GB / 22.2GB, 1.05GB/s New Data Upload : 79%|███████▉ | 4.26GB / 5.36GB, 219MB/s  ...nts/optim_065000_rank0.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_065000_rank1.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_065000_rank2.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_065000_rank4.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_065000_rank5.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_065000_rank6.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_065000_rank7.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...eckpoints/model_065000.pt: 33%|███▎ | 769MB / 2.35GB  ...nts/optim_065000_rank3.pt: 89%|████████▉ | 2.21GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.5GB / 22.2GB, 1.05GB/s New Data Upload : 79%|███████▉ | 4.30GB / 5.43GB, 218MB/s  ...nts/optim_065000_rank0.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_065000_rank1.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_065000_rank2.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_065000_rank4.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_065000_rank5.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_065000_rank6.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_065000_rank7.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...eckpoints/model_065000.pt: 33%|███▎ | 786MB / 2.35GB  ...nts/optim_065000_rank3.pt: 90%|████████▉ | 2.23GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▎ | 18.6GB / 22.2GB, 1.04GB/s New Data Upload : 80%|███████▉ | 4.34GB / 5.43GB, 218MB/s  ...nts/optim_065000_rank0.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_065000_rank1.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_065000_rank2.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_065000_rank4.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_065000_rank5.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_065000_rank6.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_065000_rank7.pt: 90%|█████████ | 2.24GB / 2.49GB  ...eckpoints/model_065000.pt: 34%|███▍ | 802MB / 2.35GB  ...nts/optim_065000_rank3.pt: 90%|█████████ | 2.24GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▍ | 18.7GB / 22.2GB, 1.03GB/s New Data Upload : 79%|███████▉ | 4.37GB / 5.50GB, 217MB/s  ...nts/optim_065000_rank0.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_065000_rank1.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_065000_rank2.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_065000_rank4.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_065000_rank5.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_065000_rank6.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_065000_rank7.pt: 91%|█████████ | 2.25GB / 2.49GB  ...eckpoints/model_065000.pt: 35%|███▌ | 826MB / 2.35GB  ...nts/optim_065000_rank3.pt: 91%|█████████ | 2.25GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▍ | 18.8GB / 22.2GB, 1.03GB/s New Data Upload : 80%|████████ | 4.41GB / 5.50GB, 217MB/s  ...nts/optim_065000_rank0.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_065000_rank1.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_065000_rank2.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_065000_rank4.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_065000_rank5.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_065000_rank6.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_065000_rank7.pt: 91%|█████████ | 2.26GB / 2.49GB  ...eckpoints/model_065000.pt: 36%|███▌ | 844MB / 2.35GB  ...nts/optim_065000_rank3.pt: 91%|█████████ | 2.26GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▌ | 18.9GB / 22.2GB, 1.02GB/s New Data Upload : 81%|████████ | 4.45GB / 5.50GB, 216MB/s  ...nts/optim_065000_rank0.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_065000_rank1.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_065000_rank2.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_065000_rank4.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_065000_rank5.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_065000_rank6.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_065000_rank7.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...eckpoints/model_065000.pt: 37%|███▋ | 863MB / 2.35GB  ...nts/optim_065000_rank3.pt: 91%|█████████▏| 2.27GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▌ | 19.0GB / 22.2GB, 1.01GB/s New Data Upload : 81%|████████ | 4.48GB / 5.57GB, 215MB/s  ...nts/optim_065000_rank0.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_065000_rank1.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_065000_rank2.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_065000_rank4.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_065000_rank5.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_065000_rank6.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_065000_rank7.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...eckpoints/model_065000.pt: 38%|███▊ | 884MB / 2.35GB  ...nts/optim_065000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▌ | 19.1GB / 22.2GB, 1.01GB/s New Data Upload : 80%|████████ | 4.52GB / 5.63GB, 215MB/s  ...nts/optim_065000_rank0.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_065000_rank1.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_065000_rank2.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_065000_rank4.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_065000_rank5.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_065000_rank6.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_065000_rank7.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...eckpoints/model_065000.pt: 39%|███▊ | 907MB / 2.35GB  ...nts/optim_065000_rank3.pt: 92%|█████████▏| 2.29GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▋ | 19.2GB / 22.2GB, 999MB/s New Data Upload : 80%|███████▉ | 4.56GB / 5.70GB, 214MB/s  ...nts/optim_065000_rank0.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_065000_rank1.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_065000_rank2.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_065000_rank4.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_065000_rank5.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_065000_rank6.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_065000_rank7.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...eckpoints/model_065000.pt: 39%|███▉ | 927MB / 2.35GB  ...nts/optim_065000_rank3.pt: 93%|█████████▎| 2.30GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.3GB / 22.2GB, 994MB/s New Data Upload : 81%|████████ | 4.59GB / 5.70GB, 213MB/s  ...nts/optim_065000_rank0.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_065000_rank1.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_065000_rank2.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_065000_rank4.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_065000_rank5.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_065000_rank6.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_065000_rank7.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...eckpoints/model_065000.pt: 40%|████ | 950MB / 2.35GB  ...nts/optim_065000_rank3.pt: 93%|█████████▎| 2.31GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.3GB / 22.2GB, 981MB/s New Data Upload : 80%|████████ | 4.63GB / 5.77GB, 212MB/s  ...nts/optim_065000_rank0.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_065000_rank1.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_065000_rank2.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_065000_rank4.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_065000_rank5.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_065000_rank6.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_065000_rank7.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...eckpoints/model_065000.pt: 41%|████▏ | 972MB / 2.35GB  ...nts/optim_065000_rank3.pt: 93%|█████████▎| 2.32GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.4GB / 22.2GB, 977MB/s New Data Upload : 81%|████████ | 4.66GB / 5.77GB, 211MB/s  ...nts/optim_065000_rank0.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_065000_rank1.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_065000_rank2.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_065000_rank4.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_065000_rank5.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_065000_rank6.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_065000_rank7.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...eckpoints/model_065000.pt: 42%|████▏ | 990MB / 2.35GB  ...nts/optim_065000_rank3.pt: 94%|█████████▎| 2.32GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.5GB / 22.2GB, 961MB/s New Data Upload : 80%|████████ | 4.70GB / 5.83GB, 209MB/s  ...nts/optim_065000_rank0.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_065000_rank1.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_065000_rank2.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_065000_rank4.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_065000_rank5.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_065000_rank6.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_065000_rank7.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...eckpoints/model_065000.pt: 43%|████▎ | 1.02GB / 2.35GB  ...nts/optim_065000_rank3.pt: 94%|█████████▎| 2.33GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.6GB / 22.2GB, 947MB/s New Data Upload : 81%|████████ | 4.73GB / 5.83GB, 208MB/s  ...nts/optim_065000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_065000_rank1.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_065000_rank2.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_065000_rank4.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_065000_rank5.pt: 94%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_065000_rank6.pt: 94%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_065000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...eckpoints/model_065000.pt: 44%|████▍ | 1.03GB / 2.35GB  ...nts/optim_065000_rank3.pt: 94%|█████████▍| 2.34GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.7GB / 22.2GB, 938MB/s New Data Upload : 82%|████████▏ | 4.76GB / 5.83GB, 207MB/s  ...nts/optim_065000_rank0.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_065000_rank1.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_065000_rank2.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_065000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_065000_rank5.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_065000_rank6.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_065000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...eckpoints/model_065000.pt: 45%|████▌ | 1.06GB / 2.35GB  ...nts/optim_065000_rank3.pt: 94%|█████████▍| 2.34GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▊ | 19.7GB / 22.2GB, 928MB/s New Data Upload : 81%|████████▏ | 4.80GB / 5.90GB, 206MB/s  ...nts/optim_065000_rank0.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_065000_rank1.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_065000_rank2.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_065000_rank4.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_065000_rank5.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_065000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_065000_rank7.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...eckpoints/model_065000.pt: 46%|████▌ | 1.08GB / 2.35GB  ...nts/optim_065000_rank3.pt: 94%|█████████▍| 2.35GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▉ | 19.8GB / 22.2GB, 914MB/s New Data Upload : 82%|████████▏ | 4.83GB / 5.90GB, 205MB/s  ...nts/optim_065000_rank0.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_065000_rank1.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_065000_rank2.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_065000_rank4.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_065000_rank5.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_065000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_065000_rank7.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...eckpoints/model_065000.pt: 47%|████▋ | 1.11GB / 2.35GB  ...nts/optim_065000_rank3.pt: 95%|█████████▍| 2.35GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▉ | 19.9GB / 22.2GB, 901MB/s New Data Upload : 81%|████████▏ | 4.86GB / 5.97GB, 204MB/s  ...nts/optim_065000_rank0.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_065000_rank1.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_065000_rank2.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_065000_rank4.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_065000_rank5.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_065000_rank6.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_065000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...eckpoints/model_065000.pt: 48%|████▊ | 1.14GB / 2.35GB  ...nts/optim_065000_rank3.pt: 95%|█████████▍| 2.36GB / 2.49GB  Processing Files (0 / 9) : 90%|████████▉ | 19.9GB / 22.2GB, 896MB/s New Data Upload : 81%|████████▏ | 4.91GB / 6.04GB, 205MB/s  ...nts/optim_065000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_065000_rank1.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_065000_rank2.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_065000_rank4.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_065000_rank5.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_065000_rank6.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_065000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...eckpoints/model_065000.pt: 50%|████▉ | 1.17GB / 2.35GB  ...nts/optim_065000_rank3.pt: 95%|█████████▌| 2.36GB / 2.49GB  Processing Files (0 / 9) : 90%|████████▉ | 20.0GB / 22.2GB, 885MB/s New Data Upload : 82%|████████▏ | 4.95GB / 6.04GB, 205MB/s  ...nts/optim_065000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_065000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_065000_rank2.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_065000_rank4.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_065000_rank5.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_065000_rank6.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_065000_rank7.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...eckpoints/model_065000.pt: 51%|█████ | 1.20GB / 2.35GB  ...nts/optim_065000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  Processing Files (0 / 9) : 90%|█████████ | 20.1GB / 22.2GB, 878MB/s New Data Upload : 83%|████████▎ | 4.99GB / 6.04GB, 206MB/s  ...nts/optim_065000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_065000_rank1.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_065000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_065000_rank4.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_065000_rank5.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_065000_rank6.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_065000_rank7.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...eckpoints/model_065000.pt: 52%|█████▏ | 1.22GB / 2.35GB  ...nts/optim_065000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.2GB / 22.2GB, 864MB/s New Data Upload : 82%|████████▏ | 5.03GB / 6.10GB, 205MB/s  ...nts/optim_065000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_065000_rank1.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_065000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_065000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_065000_rank5.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_065000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_065000_rank7.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...eckpoints/model_065000.pt: 54%|█████▎ | 1.26GB / 2.35GB  ...nts/optim_065000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.2GB / 22.2GB, 847MB/s New Data Upload : 83%|████████▎ | 5.08GB / 6.10GB, 205MB/s  ...nts/optim_065000_rank0.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_065000_rank1.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_065000_rank2.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_065000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_065000_rank5.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_065000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_065000_rank7.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...eckpoints/model_065000.pt: 55%|█████▍ | 1.29GB / 2.35GB  ...nts/optim_065000_rank3.pt: 96%|█████████▌| 2.39GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████▏| 20.3GB / 22.2GB, 836MB/s New Data Upload : 83%|████████▎ | 5.11GB / 6.17GB, 204MB/s  ...nts/optim_065000_rank0.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_065000_rank1.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_065000_rank2.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_065000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_065000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_065000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_065000_rank7.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...eckpoints/model_065000.pt: 56%|█████▋ | 1.32GB / 2.35GB  ...nts/optim_065000_rank3.pt: 96%|█████████▌| 2.39GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.4GB / 22.2GB, 823MB/s New Data Upload : 84%|████████▎ | 5.16GB / 6.17GB, 203MB/s  ...nts/optim_065000_rank0.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_065000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_065000_rank2.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_065000_rank4.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_065000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_065000_rank6.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_065000_rank7.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...eckpoints/model_065000.pt: 58%|█████▊ | 1.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 96%|█████████▋| 2.39GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.5GB / 22.2GB, 801MB/s New Data Upload : 83%|████████▎ | 5.20GB / 6.24GB, 202MB/s  ...nts/optim_065000_rank0.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_065000_rank1.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_065000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_065000_rank4.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_065000_rank5.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_065000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_065000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...eckpoints/model_065000.pt: 59%|█████▉ | 1.38GB / 2.35GB  ...nts/optim_065000_rank3.pt: 97%|█████████▋| 2.40GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.5GB / 22.2GB, 772MB/s New Data Upload : 83%|████████▎ | 5.24GB / 6.30GB, 201MB/s  ...nts/optim_065000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_065000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_065000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_065000_rank4.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_065000_rank5.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_065000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_065000_rank7.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...eckpoints/model_065000.pt: 60%|██████ | 1.41GB / 2.35GB  ...nts/optim_065000_rank3.pt: 97%|█████████▋| 2.40GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.6GB / 22.2GB, 743MB/s New Data Upload : 84%|████████▎ | 5.28GB / 6.30GB, 199MB/s  ...nts/optim_065000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_065000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_065000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_065000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_065000_rank5.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_065000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_065000_rank7.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...eckpoints/model_065000.pt: 61%|██████▏ | 1.44GB / 2.35GB  ...nts/optim_065000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.7GB / 22.2GB, 730MB/s New Data Upload : 83%|████████▎ | 5.31GB / 6.37GB, 199MB/s  ...nts/optim_065000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_065000_rank1.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_065000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_065000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_065000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_065000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_065000_rank7.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...eckpoints/model_065000.pt: 63%|██████▎ | 1.47GB / 2.35GB  ...nts/optim_065000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.7GB / 22.2GB, 700MB/s New Data Upload : 84%|████████▎ | 5.36GB / 6.40GB, 198MB/s  ...nts/optim_065000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_065000_rank1.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_065000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_065000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_065000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_065000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_065000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...eckpoints/model_065000.pt: 64%|██████▍ | 1.50GB / 2.35GB  ...nts/optim_065000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▎| 20.8GB / 22.2GB, 677MB/s New Data Upload : 84%|████████▍ | 5.39GB / 6.40GB, 197MB/s  ...nts/optim_065000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_065000_rank1.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_065000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_065000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_065000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_065000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_065000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...eckpoints/model_065000.pt: 65%|██████▍ | 1.53GB / 2.35GB  ...nts/optim_065000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 20.9GB / 22.2GB, 658MB/s New Data Upload : 85%|████████▍ | 5.43GB / 6.40GB, 198MB/s  ...nts/optim_065000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_065000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_065000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_065000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_065000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_065000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_065000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...eckpoints/model_065000.pt: 66%|██████▌ | 1.55GB / 2.35GB  ...nts/optim_065000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 20.9GB / 22.2GB, 633MB/s New Data Upload : 85%|████████▌ | 5.46GB / 6.40GB, 197MB/s  ...nts/optim_065000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_065000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_065000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_065000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_065000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_065000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_065000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...eckpoints/model_065000.pt: 67%|██████▋ | 1.57GB / 2.35GB  ...nts/optim_065000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 21.0GB / 22.2GB, 608MB/s New Data Upload : 86%|████████▌ | 5.49GB / 6.40GB, 196MB/s  ...nts/optim_065000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_065000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_065000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_065000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_065000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_065000_rank6.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_065000_rank7.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...eckpoints/model_065000.pt: 68%|██████▊ | 1.60GB / 2.35GB  ...nts/optim_065000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▍| 21.1GB / 22.2GB, 600MB/s New Data Upload : 86%|████████▋ | 5.53GB / 6.40GB, 195MB/s  ...nts/optim_065000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_065000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_065000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_065000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_065000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_065000_rank6.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_065000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...eckpoints/model_065000.pt: 69%|██████▊ | 1.61GB / 2.35GB  ...nts/optim_065000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.1GB / 22.2GB, 579MB/s New Data Upload : 87%|████████▋ | 5.55GB / 6.40GB, 193MB/s  ...nts/optim_065000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_065000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_065000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_065000_rank4.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_065000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_065000_rank6.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...nts/optim_065000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...eckpoints/model_065000.pt: 69%|██████▉ | 1.63GB / 2.35GB  ...nts/optim_065000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.2GB / 22.2GB, 565MB/s New Data Upload : 87%|████████▋ | 5.58GB / 6.40GB, 193MB/s  ...nts/optim_065000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_065000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_065000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_065000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_065000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_065000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_065000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...eckpoints/model_065000.pt: 70%|███████ | 1.65GB / 2.35GB  ...nts/optim_065000_rank3.pt: 99%|█████████▉| 2.45GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.2GB / 22.2GB, 539MB/s New Data Upload : 88%|████████▊ | 5.60GB / 6.40GB, 191MB/s  ...nts/optim_065000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_065000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_065000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_065000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_065000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_065000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_065000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...eckpoints/model_065000.pt: 71%|███████ | 1.67GB / 2.35GB  ...nts/optim_065000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 515MB/s New Data Upload : 88%|████████▊ | 5.63GB / 6.40GB, 189MB/s  ...nts/optim_065000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_065000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_065000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_065000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank7.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_065000.pt: 72%|███████▏ | 1.68GB / 2.35GB  ...nts/optim_065000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 503MB/s New Data Upload : 88%|████████▊ | 5.65GB / 6.40GB, 187MB/s  ...nts/optim_065000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_065000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_065000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_065000.pt: 72%|███████▏ | 1.70GB / 2.35GB  ...nts/optim_065000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.4GB / 22.2GB, 482MB/s New Data Upload : 89%|████████▉ | 5.68GB / 6.40GB, 185MB/s  ...nts/optim_065000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_065000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_065000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_065000.pt: 73%|███████▎ | 1.72GB / 2.35GB  ...nts/optim_065000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▋| 21.4GB / 22.2GB, 454MB/s New Data Upload : 89%|████████▉ | 5.70GB / 6.40GB, 183MB/s  ...nts/optim_065000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_065000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_065000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_065000.pt: 74%|███████▍ | 1.74GB / 2.35GB  ...nts/optim_065000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▋| 21.4GB / 22.2GB, 438MB/s New Data Upload : 89%|████████▉ | 5.72GB / 6.40GB, 181MB/s  ...nts/optim_065000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_065000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_065000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_065000.pt: 75%|███████▍ | 1.76GB / 2.35GB  ...nts/optim_065000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 417MB/s New Data Upload : 90%|████████▉ | 5.75GB / 6.40GB, 179MB/s  ...nts/optim_065000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_065000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_065000.pt: 76%|███████▌ | 1.78GB / 2.35GB  ...nts/optim_065000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 404MB/s New Data Upload : 90%|█████████ | 5.78GB / 6.40GB, 178MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_065000.pt: 77%|███████▋ | 1.80GB / 2.35GB  ...nts/optim_065000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 392MB/s New Data Upload : 91%|█████████ | 5.81GB / 6.40GB, 176MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_065000.pt: 78%|███████▊ | 1.82GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 379MB/s New Data Upload : 91%|█████████ | 5.83GB / 6.40GB, 175MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_065000.pt: 79%|███████▊ | 1.84GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 370MB/s New Data Upload : 92%|█████████▏| 5.86GB / 6.40GB, 173MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_065000.pt: 79%|███████▉ | 1.86GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.7GB / 22.2GB, 360MB/s New Data Upload : 92%|█████████▏| 5.88GB / 6.40GB, 171MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_065000.pt: 80%|████████ | 1.89GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 352MB/s New Data Upload : 92%|█████████▏| 5.91GB / 6.40GB, 170MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_065000.pt: 81%|████████▏ | 1.91GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 342MB/s New Data Upload : 93%|█████████▎| 5.93GB / 6.40GB, 168MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_065000.pt: 82%|████████▏ | 1.93GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 331MB/s New Data Upload : 93%|█████████▎| 5.96GB / 6.40GB, 166MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_065000.pt: 83%|████████▎ | 1.95GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 323MB/s New Data Upload : 93%|█████████▎| 5.98GB / 6.40GB, 165MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 84%|████████▍ | 1.98GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 314MB/s New Data Upload : 94%|█████████▍| 6.01GB / 6.40GB, 164MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 85%|████████▌ | 2.00GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 305MB/s New Data Upload : 94%|█████████▍| 6.03GB / 6.40GB, 163MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 86%|████████▌ | 2.02GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 297MB/s New Data Upload : 95%|█████████▍| 6.06GB / 6.40GB, 161MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 87%|████████▋ | 2.04GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 291MB/s New Data Upload : 95%|█████████▌| 6.08GB / 6.40GB, 160MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 88%|████████▊ | 2.06GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 285MB/s New Data Upload : 95%|█████████▌| 6.10GB / 6.40GB, 159MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 89%|████████▊ | 2.08GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 277MB/s New Data Upload : 96%|█████████▌| 6.12GB / 6.40GB, 157MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 89%|████████▉ | 2.10GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 271MB/s New Data Upload : 96%|█████████▌| 6.14GB / 6.40GB, 155MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 90%|█████████ | 2.12GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 265MB/s New Data Upload : 96%|█████████▋| 6.16GB / 6.40GB, 154MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 91%|█████████ | 2.14GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 261MB/s New Data Upload : 97%|█████████▋| 6.18GB / 6.40GB, 153MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 92%|█████████▏| 2.15GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 253MB/s New Data Upload : 97%|█████████▋| 6.20GB / 6.40GB, 151MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 92%|█████████▏| 2.17GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 248MB/s New Data Upload : 97%|█████████▋| 6.22GB / 6.40GB, 149MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 93%|█████████▎| 2.19GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 243MB/s New Data Upload : 97%|█████████▋| 6.24GB / 6.40GB, 148MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 94%|█████████▍| 2.20GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 238MB/s New Data Upload : 98%|█████████▊| 6.25GB / 6.40GB, 146MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 94%|█████████▍| 2.22GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 233MB/s New Data Upload : 98%|█████████▊| 6.26GB / 6.40GB, 143MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 95%|█████████▍| 2.23GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 227MB/s New Data Upload : 98%|█████████▊| 6.28GB / 6.40GB, 141MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 95%|█████████▌| 2.23GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 222MB/s New Data Upload : 98%|█████████▊| 6.28GB / 6.40GB, 139MB/s  ...nts/optim_065000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 96%|█████████▌| 2.24GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 214MB/s New Data Upload : 98%|█████████▊| 6.29GB / 6.40GB, 135MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 96%|█████████▌| 2.25GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 209MB/s New Data Upload : 98%|█████████▊| 6.30GB / 6.40GB, 133MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 96%|█████████▋| 2.26GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 202MB/s New Data Upload : 99%|█████████▊| 6.31GB / 6.40GB, 129MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 194MB/s New Data Upload : 99%|█████████▉| 6.32GB / 6.40GB, 126MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 188MB/s New Data Upload : 99%|█████████▉| 6.33GB / 6.40GB, 123MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 98%|█████████▊| 2.29GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 183MB/s New Data Upload : 99%|█████████▉| 6.34GB / 6.40GB, 121MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 175MB/s New Data Upload : 99%|█████████▉| 6.35GB / 6.40GB, 117MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 170MB/s New Data Upload : 99%|█████████▉| 6.35GB / 6.40GB, 114MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 163MB/s New Data Upload : 99%|█████████▉| 6.36GB / 6.40GB, 110MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 156MB/s New Data Upload : 99%|█████████▉| 6.36GB / 6.40GB, 106MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 151MB/s New Data Upload : 99%|█████████▉| 6.37GB / 6.40GB, 103MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 143MB/s New Data Upload : 100%|█████████▉| 6.37GB / 6.40GB, 99.0MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 138MB/s New Data Upload : 100%|█████████▉| 6.37GB / 6.40GB, 96.0MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 130MB/s New Data Upload : 100%|█████████▉| 6.37GB / 6.40GB, 92.5MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 117MB/s New Data Upload : 100%|█████████▉| 6.37GB / 6.40GB, 86.3MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 106MB/s New Data Upload : 100%|█████████▉| 6.37GB / 6.40GB, 80.5MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 99.6MB/s New Data Upload : 100%|█████████▉| 6.37GB / 6.40GB, 78.3MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 94.6MB/s New Data Upload : 100%|█████████▉| 6.38GB / 6.40GB, 76.1MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 85.0MB/s New Data Upload : 100%|█████████▉| 6.38GB / 6.40GB, 71.6MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 77.6MB/s New Data Upload : 100%|█████████▉| 6.38GB / 6.40GB, 66.5MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 72.2MB/s New Data Upload : 100%|█████████▉| 6.38GB / 6.40GB, 61.8MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 64.4MB/s New Data Upload : 100%|█████████▉| 6.38GB / 6.40GB, 56.2MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 61.8MB/s New Data Upload : 100%|█████████▉| 6.38GB / 6.40GB, 53.9MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 55.5MB/s New Data Upload : 100%|█████████▉| 6.39GB / 6.40GB, 49.4MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 99%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 49.2MB/s New Data Upload : 100%|█████████▉| 6.39GB / 6.40GB, 44.3MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 46.7MB/s New Data Upload : 100%|█████████▉| 6.39GB / 6.40GB, 42.3MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 40.4MB/s New Data Upload : 100%|█████████▉| 6.39GB / 6.40GB, 37.5MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 34.7MB/s New Data Upload : 100%|█████████▉| 6.39GB / 6.40GB, 33.0MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 32.2MB/s New Data Upload : 100%|█████████▉| 6.39GB / 6.40GB, 30.9MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 27.2MB/s New Data Upload : 100%|█████████▉| 6.39GB / 6.40GB, 26.7MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 23.2MB/s New Data Upload : 100%|█████████▉| 6.40GB / 6.40GB, 22.7MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (7 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 19.7MB/s New Data Upload : 100%|█████████▉| 6.40GB / 6.40GB, 19.4MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 10.4MB/s New Data Upload : 100%|██████████| 6.40GB / 6.40GB, 10.4MB/s  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_065000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 2.62MB/s New Data Upload : 100%|██████████| 6.40GB / 6.40GB, 2.62MB/s ...nts/optim_065000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_065000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_065000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_065000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_065000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_065000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_065000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB ...eckpoints/model_065000.pt: 100%|██████████| 2.35GB / 2.35GB ...nts/optim_065000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB [19:02:24] Step 65000 uploaded [19:57:24] Uploading step 70000... Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...eckpoints/model_070000.pt: 0%| | 583kB / 2.35GB  ...eckpoints/model_070000.pt: 0%| | 583kB / 2.35GB  Processing Files (0 / 1) : 0%| | 583kB / 22.2GB, 728kB/s New Data Upload : 0%| | 583kB / 537MB, 728kB/s  ...eckpoints/model_070000.pt: 0%| | 3.66MB / 2.35GB  Processing Files (0 / 1) : 0%| | 3.66MB / 22.2GB, 3.66MB/s ...eckpoints/model_070000.pt: 0%| | 4.25MB / 2.35GB  Processing Files (0 / 1) : 0%| | 4.25MB / 22.2GB, 3.54MB/s New Data Upload : 0%| | 1.17MB / 604MB, 971kB/s  ...eckpoints/model_070000.pt: 0%| | 5.41MB / 2.35GB  Processing Files (0 / 1) : 0%| | 5.41MB / 22.2GB, 3.87MB/s New Data Upload : 0%| | 2.33MB / 604MB, 1.67MB/s  ...eckpoints/model_070000.pt: 0%| | 8.33MB / 2.35GB  Processing Files (0 / 1) : 0%| | 8.33MB / 22.2GB, 5.20MB/s New Data Upload : 1%| | 5.25MB / 604MB, 3.28MB/s  ...eckpoints/model_070000.pt: 0%| | 11.2MB / 2.35GB  Processing Files (0 / 1) : 0%| | 11.2MB / 22.2GB, 6.24MB/s New Data Upload : 1%|▏ | 8.16MB / 604MB, 4.53MB/s  ...eckpoints/model_070000.pt: 1%| | 15.9MB / 2.35GB  Processing Files (0 / 1) : 0%| | 15.9MB / 22.2GB, 7.95MB/s New Data Upload : 2%|▏ | 12.8MB / 604MB, 6.41MB/s  ...eckpoints/model_070000.pt: 1%| | 18.2MB / 2.35GB  Processing Files (0 / 1) : 0%| | 18.2MB / 22.2GB, 8.29MB/s New Data Upload : 3%|▎ | 15.2MB / 604MB, 6.89MB/s  ...eckpoints/model_070000.pt: 1%| | 22.9MB / 2.35GB  Processing Files (0 / 1) : 0%| | 22.9MB / 22.2GB, 9.54MB/s New Data Upload : 3%|▎ | 19.8MB / 604MB, 8.26MB/s  ...eckpoints/model_070000.pt: 1%| | 25.8MB / 2.35GB  Processing Files (0 / 1) : 0%| | 25.8MB / 22.2GB, 9.93MB/s New Data Upload : 4%|▍ | 22.7MB / 604MB, 8.74MB/s  ...eckpoints/model_070000.pt: 1%|▏ | 30.5MB / 2.35GB  Processing Files (0 / 1) : 0%| | 30.5MB / 22.2GB, 10.9MB/s New Data Upload : 5%|▍ | 27.4MB / 604MB, 9.79MB/s  ...eckpoints/model_070000.pt: 2%|▏ | 35.7MB / 2.35GB  Processing Files (0 / 1) : 0%| | 35.7MB / 22.2GB, 11.9MB/s New Data Upload : 5%|▌ | 32.6MB / 604MB, 10.9MB/s  ...eckpoints/model_070000.pt: 2%|▏ | 38.6MB / 2.35GB  Processing Files (0 / 1) : 0%| | 38.6MB / 22.2GB, 12.1MB/s New Data Upload : 6%|▌ | 35.6MB / 604MB, 11.1MB/s  ...eckpoints/model_070000.pt: 2%|▏ | 44.5MB / 2.35GB  Processing Files (0 / 1) : 0%| | 44.5MB / 22.2GB, 13.1MB/s New Data Upload : 7%|▋ | 41.4MB / 604MB, 12.2MB/s  ...eckpoints/model_070000.pt: 2%|▏ | 50.3MB / 2.35GB  Processing Files (0 / 1) : 0%| | 50.3MB / 22.2GB, 14.0MB/s New Data Upload : 8%|▊ | 47.2MB / 604MB, 13.1MB/s  ...eckpoints/model_070000.pt: 2%|▏ | 56.1MB / 2.35GB  Processing Files (0 / 1) : 0%| | 56.1MB / 22.2GB, 14.8MB/s New Data Upload : 9%|▉ | 53.0MB / 604MB, 14.0MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 62.5MB / 2.35GB  Processing Files (0 / 1) : 0%| | 62.5MB / 22.2GB, 15.6MB/s New Data Upload : 10%|▉ | 59.5MB / 604MB, 14.9MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 65.5MB / 2.35GB  Processing Files (0 / 1) : 0%| | 65.5MB / 22.2GB, 15.6MB/s New Data Upload : 10%|█ | 62.4MB / 604MB, 14.8MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 69.5MB / 2.35GB  Processing Files (0 / 1) : 0%| | 69.5MB / 22.2GB, 15.8MB/s New Data Upload : 11%|█ | 66.5MB / 604MB, 15.1MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 69.5MB / 2.35GB  ...eckpoints/model_070000.pt: 3%|▎ | 69.5MB / 2.35GB  ...eckpoints/model_070000.pt: 3%|▎ | 69.5MB / 2.35GB  ...eckpoints/model_070000.pt: 3%|▎ | 69.5MB / 2.35GB  ...eckpoints/model_070000.pt: 3%|▎ | 70.1MB / 2.35GB  Processing Files (0 / 1) : 0%| | 70.1MB / 22.2GB, 13.0MB/s New Data Upload : 11%|█ | 67.0MB / 604MB, 12.4MB/s  ...nts/optim_070000_rank1.pt: 0%| | 591kB / 2.49GB  ...nts/optim_070000_rank2.pt: 0%| | 591kB / 2.49GB  ...nts/optim_070000_rank3.pt: 0%| | 1.15MB / 2.49GB  ...nts/optim_070000_rank4.pt: 0%| | 870kB / 2.49GB  ...nts/optim_070000_rank5.pt: 0%| | 800kB / 2.49GB  ...nts/optim_070000_rank6.pt: 0%| | 2.72MB / 2.49GB  ...nts/optim_070000_rank7.pt: 0%| | 591kB / 2.49GB  ...eckpoints/model_070000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_070000_rank1.pt: 0%| | 591kB / 2.49GB  ...nts/optim_070000_rank2.pt: 0%| | 591kB / 2.49GB  ...nts/optim_070000_rank3.pt: 0%| | 1.15MB / 2.49GB  ...nts/optim_070000_rank4.pt: 0%| | 870kB / 2.49GB  ...nts/optim_070000_rank5.pt: 0%| | 800kB / 2.49GB  ...nts/optim_070000_rank6.pt: 0%| | 2.72MB / 2.49GB  ...nts/optim_070000_rank7.pt: 0%| | 591kB / 2.49GB  Processing Files (0 / 8) : 0%| | 77.4MB / 22.2GB, 13.8MB/s New Data Upload : 12%|█▏ | 70.0MB / 604MB, 12.5MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_070000_rank1.pt: 0%| | 591kB / 2.49GB  ...nts/optim_070000_rank2.pt: 0%| | 591kB / 2.49GB  ...nts/optim_070000_rank3.pt: 0%| | 1.15MB / 2.49GB  ...nts/optim_070000_rank4.pt: 0%| | 870kB / 2.49GB  ...nts/optim_070000_rank5.pt: 0%| | 800kB / 2.49GB  ...nts/optim_070000_rank6.pt: 0%| | 2.72MB / 2.49GB  ...nts/optim_070000_rank7.pt: 0%| | 591kB / 2.49GB  ...eckpoints/model_070000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_070000_rank1.pt: 0%| | 967kB / 2.49GB  ...nts/optim_070000_rank2.pt: 0%| | 967kB / 2.49GB  ...nts/optim_070000_rank3.pt: 0%| | 1.53MB / 2.49GB  ...nts/optim_070000_rank4.pt: 0%| | 1.25MB / 2.49GB  ...nts/optim_070000_rank5.pt: 0%| | 1.39MB / 2.49GB  ...nts/optim_070000_rank6.pt: 0%| | 3.10MB / 2.49GB  ...nts/optim_070000_rank7.pt: 0%| | 967kB / 2.49GB  Processing Files (0 / 8) : 0%| | 80.3MB / 22.2GB, 13.4MB/s New Data Upload : 10%|▉ | 70.6MB / 738MB, 11.8MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_070000_rank1.pt: 0%| | 3.05MB / 2.49GB  ...nts/optim_070000_rank2.pt: 0%| | 3.05MB / 2.49GB  ...nts/optim_070000_rank3.pt: 0%| | 4.02MB / 2.49GB  ...nts/optim_070000_rank4.pt: 0%| | 3.55MB / 2.49GB  ...nts/optim_070000_rank5.pt: 0%| | 4.51MB / 2.49GB  ...nts/optim_070000_rank6.pt: 0%| | 6.77MB / 2.49GB  ...nts/optim_070000_rank7.pt: 0%| | 3.59MB / 2.49GB  Processing Files (0 / 8) : 0%| | 98.7MB / 22.2GB, 15.9MB/s New Data Upload : 10%|█ | 75.8MB / 738MB, 12.2MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_070000_rank1.pt: 0%| | 6.05MB / 2.49GB  ...nts/optim_070000_rank2.pt: 0%| | 6.05MB / 2.49GB  ...nts/optim_070000_rank3.pt: 0%| | 8.01MB / 2.49GB  ...nts/optim_070000_rank4.pt: 0%| | 7.05MB / 2.49GB  ...nts/optim_070000_rank5.pt: 0%| | 8.99MB / 2.49GB  ...nts/optim_070000_rank6.pt: 1%| | 13.5MB / 2.49GB  ...nts/optim_070000_rank7.pt: 0%| | 9.62MB / 2.49GB  Processing Files (0 / 8) : 1%| | 129MB / 22.2GB, 20.2MB/s New Data Upload : 10%|█ | 84.0MB / 805MB, 13.1MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_070000_rank1.pt: 0%| | 9.73MB / 2.49GB  ...nts/optim_070000_rank2.pt: 1%| | 13.8MB / 2.49GB  ...nts/optim_070000_rank3.pt: 1%| | 13.4MB / 2.49GB  ...nts/optim_070000_rank4.pt: 0%| | 11.6MB / 2.49GB  ...nts/optim_070000_rank5.pt: 1%| | 14.4MB / 2.49GB  ...nts/optim_070000_rank6.pt: 1%| | 23.1MB / 2.49GB  ...nts/optim_070000_rank7.pt: 1%| | 14.4MB / 2.49GB  Processing Files (0 / 8) : 1%| | 170MB / 22.2GB, 25.8MB/s New Data Upload : 12%|█▏ | 100MB / 805MB, 15.2MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_070000_rank1.pt: 1%| | 12.6MB / 2.49GB  ...nts/optim_070000_rank2.pt: 1%| | 19.8MB / 2.49GB  ...nts/optim_070000_rank3.pt: 1%| | 17.4MB / 2.49GB  ...nts/optim_070000_rank4.pt: 1%| | 15.1MB / 2.49GB  ...nts/optim_070000_rank5.pt: 1%| | 18.5MB / 2.49GB  ...nts/optim_070000_rank6.pt: 1%| | 30.2MB / 2.49GB  ...nts/optim_070000_rank7.pt: 1%| | 20.6MB / 2.49GB  Processing Files (0 / 8) : 1%| | 204MB / 22.2GB, 30.1MB/s New Data Upload : 14%|█▍ | 112MB / 805MB, 16.5MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_070000_rank1.pt: 1%| | 18.0MB / 2.49GB  ...nts/optim_070000_rank2.pt: 1%| | 25.1MB / 2.49GB  ...nts/optim_070000_rank3.pt: 1%| | 24.4MB / 2.49GB  ...nts/optim_070000_rank4.pt: 1%| | 21.4MB / 2.49GB  ...nts/optim_070000_rank5.pt: 1%| | 26.4MB / 2.49GB  ...nts/optim_070000_rank6.pt: 2%|▏ | 42.0MB / 2.49GB  ...nts/optim_070000_rank7.pt: 1%| | 28.7MB / 2.49GB  Processing Files (0 / 8) : 1%| | 256MB / 22.2GB, 36.6MB/s New Data Upload : 15%|█▍ | 129MB / 872MB, 18.4MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_070000_rank1.pt: 1%| | 26.7MB / 2.49GB  ...nts/optim_070000_rank2.pt: 1%|▏ | 36.0MB / 2.49GB  ...nts/optim_070000_rank3.pt: 1%| | 29.7MB / 2.49GB  ...nts/optim_070000_rank4.pt: 1%| | 26.7MB / 2.49GB  ...nts/optim_070000_rank5.pt: 1%|▏ | 34.1MB / 2.49GB  ...nts/optim_070000_rank6.pt: 2%|▏ | 47.1MB / 2.49GB  ...nts/optim_070000_rank7.pt: 2%|▏ | 41.5MB / 2.49GB  Processing Files (0 / 8) : 1%|▏ | 312MB / 22.2GB, 43.3MB/s New Data Upload : 17%|█▋ | 151MB / 872MB, 20.9MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_070000_rank1.pt: 1%|▏ | 32.2MB / 2.49GB  ...nts/optim_070000_rank2.pt: 2%|▏ | 45.0MB / 2.49GB  ...nts/optim_070000_rank3.pt: 1%|▏ | 36.9MB / 2.49GB  ...nts/optim_070000_rank4.pt: 1%|▏ | 33.2MB / 2.49GB  ...nts/optim_070000_rank5.pt: 2%|▏ | 42.1MB / 2.49GB  ...nts/optim_070000_rank6.pt: 2%|▏ | 58.9MB / 2.49GB  ...nts/optim_070000_rank7.pt: 2%|▏ | 51.3MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 370MB / 22.2GB, 50.0MB/s New Data Upload : 18%|█▊ | 173MB / 939MB, 23.4MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_070000_rank1.pt: 2%|▏ | 38.1MB / 2.49GB  ...nts/optim_070000_rank2.pt: 2%|▏ | 53.2MB / 2.49GB  ...nts/optim_070000_rank3.pt: 2%|▏ | 43.7MB / 2.49GB  ...nts/optim_070000_rank4.pt: 2%|▏ | 39.3MB / 2.49GB  ...nts/optim_070000_rank5.pt: 2%|▏ | 50.1MB / 2.49GB  ...nts/optim_070000_rank6.pt: 3%|▎ | 69.1MB / 2.49GB  ...nts/optim_070000_rank7.pt: 2%|▏ | 61.6MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 425MB / 22.2GB, 56.0MB/s New Data Upload : 21%|██ | 194MB / 939MB, 25.5MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_070000_rank1.pt: 2%|▏ | 44.8MB / 2.49GB  ...nts/optim_070000_rank2.pt: 2%|▏ | 61.1MB / 2.49GB  ...nts/optim_070000_rank3.pt: 2%|▏ | 48.9MB / 2.49GB  ...nts/optim_070000_rank4.pt: 2%|▏ | 47.5MB / 2.49GB  ...nts/optim_070000_rank5.pt: 2%|▏ | 56.3MB / 2.49GB  ...nts/optim_070000_rank6.pt: 3%|▎ | 77.4MB / 2.49GB  ...nts/optim_070000_rank7.pt: 3%|▎ | 69.3MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 476MB / 22.2GB, 61.0MB/s New Data Upload : 23%|██▎ | 215MB / 939MB, 27.6MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_070000_rank1.pt: 2%|▏ | 53.8MB / 2.49GB  ...nts/optim_070000_rank2.pt: 3%|▎ | 70.0MB / 2.49GB  ...nts/optim_070000_rank3.pt: 2%|▏ | 55.3MB / 2.49GB  ...nts/optim_070000_rank4.pt: 2%|▏ | 53.4MB / 2.49GB  ...nts/optim_070000_rank5.pt: 3%|▎ | 64.4MB / 2.49GB  ...nts/optim_070000_rank6.pt: 3%|▎ | 86.8MB / 2.49GB  ...nts/optim_070000_rank7.pt: 3%|▎ | 78.7MB / 2.49GB  Processing Files (0 / 8) : 2%|▏ | 533MB / 22.2GB, 66.6MB/s New Data Upload : 25%|██▌ | 239MB / 939MB, 29.9MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_070000_rank1.pt: 3%|▎ | 62.4MB / 2.49GB  ...nts/optim_070000_rank2.pt: 3%|▎ | 77.4MB / 2.49GB  ...nts/optim_070000_rank3.pt: 3%|▎ | 62.3MB / 2.49GB  ...nts/optim_070000_rank4.pt: 2%|▏ | 60.6MB / 2.49GB  ...nts/optim_070000_rank5.pt: 3%|▎ | 73.8MB / 2.49GB  ...nts/optim_070000_rank6.pt: 4%|▍ | 94.9MB / 2.49GB  ...nts/optim_070000_rank7.pt: 3%|▎ | 86.6MB / 2.49GB  Processing Files (0 / 8) : 3%|▎ | 588MB / 22.2GB, 71.7MB/s New Data Upload : 27%|██▋ | 258MB / 939MB, 31.5MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_070000_rank1.pt: 3%|▎ | 74.6MB / 2.49GB  ...nts/optim_070000_rank2.pt: 3%|▎ | 85.5MB / 2.49GB  ...nts/optim_070000_rank3.pt: 3%|▎ | 71.1MB / 2.49GB  ...nts/optim_070000_rank4.pt: 3%|▎ | 68.6MB / 2.49GB  ...nts/optim_070000_rank5.pt: 3%|▎ | 81.1MB / 2.49GB  ...nts/optim_070000_rank6.pt: 4%|▍ | 104MB / 2.49GB  ...nts/optim_070000_rank7.pt: 4%|▍ | 94.9MB / 2.49GB  Processing Files (0 / 8) : 3%|▎ | 650MB / 22.2GB, 77.4MB/s New Data Upload : 29%|██▊ | 289MB / 1.01GB, 34.4MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_070000_rank1.pt: 3%|▎ | 85.0MB / 2.49GB  ...nts/optim_070000_rank2.pt: 4%|▎ | 88.8MB / 2.49GB  ...nts/optim_070000_rank3.pt: 3%|▎ | 73.9MB / 2.49GB  ...nts/optim_070000_rank4.pt: 3%|▎ | 73.2MB / 2.49GB  ...nts/optim_070000_rank5.pt: 3%|▎ | 84.9MB / 2.49GB  ...nts/optim_070000_rank6.pt: 4%|▍ | 108MB / 2.49GB  ...nts/optim_070000_rank7.pt: 4%|▍ | 101MB / 2.49GB  Processing Files (0 / 8) : 3%|▎ | 684MB / 22.2GB, 79.5MB/s New Data Upload : 31%|███ | 310MB / 1.01GB, 36.1MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 70.7MB / 2.35GB  ...nts/optim_070000_rank1.pt: 4%|▍ | 95.8MB / 2.49GB  ...nts/optim_070000_rank2.pt: 4%|▎ | 92.6MB / 2.49GB  ...nts/optim_070000_rank3.pt: 3%|▎ | 79.5MB / 2.49GB  ...nts/optim_070000_rank4.pt: 3%|▎ | 82.9MB / 2.49GB  ...nts/optim_070000_rank5.pt: 4%|▎ | 90.9MB / 2.49GB  ...nts/optim_070000_rank6.pt: 5%|▍ | 115MB / 2.49GB  ...nts/optim_070000_rank7.pt: 4%|▍ | 108MB / 2.49GB  Processing Files (0 / 8) : 3%|▎ | 735MB / 22.2GB, 83.5MB/s New Data Upload : 30%|██▉ | 339MB / 1.14GB, 38.6MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 70.7MB / 2.35GB  ...nts/optim_070000_rank1.pt: 4%|▍ | 108MB / 2.49GB  ...nts/optim_070000_rank2.pt: 4%|▍ | 96.9MB / 2.49GB  ...nts/optim_070000_rank3.pt: 3%|▎ | 86.8MB / 2.49GB  ...nts/optim_070000_rank4.pt: 4%|▎ | 92.9MB / 2.49GB  ...nts/optim_070000_rank5.pt: 4%|▍ | 97.9MB / 2.49GB  ...nts/optim_070000_rank6.pt: 5%|▍ | 122MB / 2.49GB  ...nts/optim_070000_rank7.pt: 5%|▍ | 113MB / 2.49GB  Processing Files (0 / 8) : 4%|▎ | 788MB / 22.2GB, 87.5MB/s New Data Upload : 32%|███▏ | 367MB / 1.14GB, 40.8MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 71.2MB / 2.35GB  ...nts/optim_070000_rank1.pt: 5%|▍ | 116MB / 2.49GB  ...nts/optim_070000_rank2.pt: 4%|▍ | 102MB / 2.49GB  ...nts/optim_070000_rank3.pt: 4%|▍ | 96.6MB / 2.49GB  ...nts/optim_070000_rank4.pt: 4%|▍ | 106MB / 2.49GB  ...nts/optim_070000_rank5.pt: 4%|▍ | 109MB / 2.49GB  ...nts/optim_070000_rank6.pt: 5%|▌ | 131MB / 2.49GB  ...nts/optim_070000_rank7.pt: 5%|▍ | 123MB / 2.49GB  Processing Files (0 / 8) : 4%|▍ | 856MB / 22.2GB, 93.0MB/s New Data Upload : 35%|███▍ | 398MB / 1.14GB, 43.2MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 75.2MB / 2.35GB  ...nts/optim_070000_rank1.pt: 5%|▌ | 127MB / 2.49GB  ...nts/optim_070000_rank2.pt: 4%|▍ | 111MB / 2.49GB  ...nts/optim_070000_rank3.pt: 4%|▍ | 110MB / 2.49GB  ...nts/optim_070000_rank4.pt: 5%|▍ | 116MB / 2.49GB  ...nts/optim_070000_rank5.pt: 5%|▍ | 120MB / 2.49GB  ...nts/optim_070000_rank6.pt: 6%|▌ | 141MB / 2.49GB  ...nts/optim_070000_rank7.pt: 5%|▌ | 129MB / 2.49GB  Processing Files (0 / 8) : 4%|▍ | 930MB / 22.2GB, 98.9MB/s New Data Upload : 38%|███▊ | 431MB / 1.14GB, 45.9MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 75.2MB / 2.35GB  ...nts/optim_070000_rank1.pt: 6%|▌ | 137MB / 2.49GB  ...nts/optim_070000_rank2.pt: 5%|▍ | 119MB / 2.49GB  ...nts/optim_070000_rank3.pt: 5%|▍ | 122MB / 2.49GB  ...nts/optim_070000_rank4.pt: 5%|▌ | 128MB / 2.49GB  ...nts/optim_070000_rank5.pt: 5%|▌ | 129MB / 2.49GB  ...nts/optim_070000_rank6.pt: 6%|▌ | 150MB / 2.49GB  ...nts/optim_070000_rank7.pt: 6%|▌ | 139MB / 2.49GB  Processing Files (0 / 8) : 4%|▍ | 999MB / 22.2GB, 104MB/s New Data Upload : 41%|████ | 463MB / 1.14GB, 48.3MB/s  ...eckpoints/model_070000.pt: 3%|▎ | 82.0MB / 2.35GB  ...nts/optim_070000_rank1.pt: 6%|▌ | 150MB / 2.49GB  ...nts/optim_070000_rank2.pt: 5%|▌ | 129MB / 2.49GB  ...nts/optim_070000_rank3.pt: 5%|▌ | 135MB / 2.49GB  ...nts/optim_070000_rank4.pt: 6%|▌ | 145MB / 2.49GB  ...nts/optim_070000_rank5.pt: 6%|▌ | 142MB / 2.49GB  ...nts/optim_070000_rank6.pt: 7%|▋ | 165MB / 2.49GB  ...nts/optim_070000_rank7.pt: 6%|▌ | 152MB / 2.49GB  Processing Files (0 / 8) : 5%|▍ | 1.10GB / 22.2GB, 112MB/s New Data Upload : 42%|████▏ | 506MB / 1.21GB, 51.7MB/s  ...eckpoints/model_070000.pt: 4%|▍ | 90.5MB / 2.35GB  ...nts/optim_070000_rank1.pt: 6%|▋ | 159MB / 2.49GB  ...nts/optim_070000_rank2.pt: 6%|▌ | 140MB / 2.49GB  ...nts/optim_070000_rank3.pt: 6%|▌ | 152MB / 2.49GB  ...nts/optim_070000_rank4.pt: 7%|▋ | 163MB / 2.49GB  ...nts/optim_070000_rank5.pt: 6%|▋ | 156MB / 2.49GB  ...nts/optim_070000_rank6.pt: 7%|▋ | 178MB / 2.49GB  ...nts/optim_070000_rank7.pt: 7%|▋ | 162MB / 2.49GB  Processing Files (0 / 8) : 5%|▌ | 1.20GB / 22.2GB, 120MB/s New Data Upload : 43%|████▎ | 547MB / 1.27GB, 54.7MB/s  ...eckpoints/model_070000.pt: 4%|▍ | 99.0MB / 2.35GB  ...nts/optim_070000_rank1.pt: 7%|▋ | 167MB / 2.49GB  ...nts/optim_070000_rank2.pt: 6%|▌ | 151MB / 2.49GB  ...nts/optim_070000_rank3.pt: 7%|▋ | 167MB / 2.49GB  ...nts/optim_070000_rank4.pt: 7%|▋ | 174MB / 2.49GB  ...nts/optim_070000_rank5.pt: 7%|▋ | 168MB / 2.49GB  ...nts/optim_070000_rank6.pt: 8%|▊ | 187MB / 2.49GB  ...nts/optim_070000_rank7.pt: 7%|▋ | 171MB / 2.49GB  Processing Files (0 / 8) : 6%|▌ | 1.28GB / 22.2GB, 126MB/s New Data Upload : 45%|████▌ | 579MB / 1.27GB, 56.8MB/s  ...eckpoints/model_070000.pt: 5%|▍ | 108MB / 2.35GB  ...nts/optim_070000_rank1.pt: 7%|▋ | 184MB / 2.49GB  ...nts/optim_070000_rank2.pt: 7%|▋ | 170MB / 2.49GB  ...nts/optim_070000_rank3.pt: 8%|▊ | 190MB / 2.49GB  ...nts/optim_070000_rank4.pt: 8%|▊ | 190MB / 2.49GB  ...nts/optim_070000_rank5.pt: 8%|▊ | 187MB / 2.49GB  ...nts/optim_070000_rank6.pt: 8%|▊ | 201MB / 2.49GB  ...nts/optim_070000_rank7.pt: 8%|▊ | 187MB / 2.49GB  Processing Files (0 / 8) : 6%|▋ | 1.42GB / 22.2GB, 139MB/s New Data Upload : 49%|████▊ | 619MB / 1.27GB, 60.7MB/s  ...eckpoints/model_070000.pt: 5%|▍ | 115MB / 2.35GB  ...nts/optim_070000_rank1.pt: 8%|▊ | 201MB / 2.49GB  ...nts/optim_070000_rank2.pt: 8%|▊ | 188MB / 2.49GB  ...nts/optim_070000_rank3.pt: 8%|▊ | 206MB / 2.49GB  ...nts/optim_070000_rank4.pt: 8%|▊ | 207MB / 2.49GB  ...nts/optim_070000_rank5.pt: 8%|▊ | 204MB / 2.49GB  ...nts/optim_070000_rank6.pt: 9%|▊ | 214MB / 2.49GB  ...nts/optim_070000_rank7.pt: 8%|▊ | 204MB / 2.49GB  Processing Files (0 / 8) : 7%|▋ | 1.54GB / 22.2GB, 151MB/s New Data Upload : 51%|█████ | 652MB / 1.27GB, 63.9MB/s  ...eckpoints/model_070000.pt: 5%|▌ | 119MB / 2.35GB  ...nts/optim_070000_rank1.pt: 9%|▉ | 224MB / 2.49GB  ...nts/optim_070000_rank2.pt: 9%|▊ | 212MB / 2.49GB  ...nts/optim_070000_rank3.pt: 9%|▉ | 228MB / 2.49GB  ...nts/optim_070000_rank4.pt: 9%|▉ | 229MB / 2.49GB  ...nts/optim_070000_rank5.pt: 9%|▉ | 227MB / 2.49GB  ...nts/optim_070000_rank6.pt: 9%|▉ | 234MB / 2.49GB  ...nts/optim_070000_rank7.pt: 9%|▉ | 227MB / 2.49GB  Processing Files (0 / 8) : 8%|▊ | 1.70GB / 22.2GB, 167MB/s New Data Upload : 51%|█████▏ | 689MB / 1.34GB, 67.5MB/s  ...eckpoints/model_070000.pt: 5%|▌ | 128MB / 2.35GB  ...nts/optim_070000_rank1.pt: 10%|█ | 255MB / 2.49GB  ...nts/optim_070000_rank2.pt: 10%|▉ | 245MB / 2.49GB  ...nts/optim_070000_rank3.pt: 10%|█ | 258MB / 2.49GB  ...nts/optim_070000_rank4.pt: 10%|█ | 260MB / 2.49GB  ...nts/optim_070000_rank5.pt: 10%|█ | 257MB / 2.49GB  ...nts/optim_070000_rank6.pt: 11%|█ | 265MB / 2.49GB  ...nts/optim_070000_rank7.pt: 10%|█ | 258MB / 2.49GB  Processing Files (0 / 8) : 9%|▊ | 1.93GB / 22.2GB, 189MB/s New Data Upload : 52%|█████▏ | 737MB / 1.41GB, 72.2MB/s  ...eckpoints/model_070000.pt: 6%|▌ | 135MB / 2.35GB  ...nts/optim_070000_rank1.pt: 11%|█▏ | 282MB / 2.49GB  ...nts/optim_070000_rank2.pt: 11%|█ | 273MB / 2.49GB  ...nts/optim_070000_rank3.pt: 11%|█▏ | 286MB / 2.49GB  ...nts/optim_070000_rank4.pt: 12%|█▏ | 287MB / 2.49GB  ...nts/optim_070000_rank5.pt: 11%|█▏ | 285MB / 2.49GB  ...nts/optim_070000_rank6.pt: 12%|█▏ | 289MB / 2.49GB  ...nts/optim_070000_rank7.pt: 11%|█▏ | 285MB / 2.49GB  Processing Files (0 / 8) : 10%|▉ | 2.12GB / 22.2GB, 208MB/s New Data Upload : 50%|█████ | 777MB / 1.54GB, 76.2MB/s  ...eckpoints/model_070000.pt: 6%|▌ | 138MB / 2.35GB  ...nts/optim_070000_rank1.pt: 13%|█▎ | 319MB / 2.49GB  ...nts/optim_070000_rank2.pt: 13%|█▎ | 314MB / 2.49GB  ...nts/optim_070000_rank3.pt: 13%|█▎ | 322MB / 2.49GB  ...nts/optim_070000_rank4.pt: 13%|█▎ | 323MB / 2.49GB  ...nts/optim_070000_rank5.pt: 13%|█▎ | 321MB / 2.49GB  ...nts/optim_070000_rank6.pt: 13%|█▎ | 323MB / 2.49GB  ...nts/optim_070000_rank7.pt: 13%|█▎ | 322MB / 2.49GB  Processing Files (0 / 8) : 11%|█ | 2.38GB / 22.2GB, 233MB/s New Data Upload : 51%|█████▏ | 828MB / 1.61GB, 81.1MB/s  ...eckpoints/model_070000.pt: 6%|▌ | 141MB / 2.35GB  ...nts/optim_070000_rank1.pt: 13%|█▎ | 335MB / 2.49GB  ...nts/optim_070000_rank2.pt: 13%|█▎ | 329MB / 2.49GB  ...nts/optim_070000_rank3.pt: 14%|█▎ | 337MB / 2.49GB  ...nts/optim_070000_rank4.pt: 14%|█▎ | 339MB / 2.49GB  ...nts/optim_070000_rank5.pt: 14%|█▎ | 336MB / 2.49GB  ...nts/optim_070000_rank6.pt: 14%|█▎ | 339MB / 2.49GB  ...nts/optim_070000_rank7.pt: 14%|█▎ | 338MB / 2.49GB  Processing Files (0 / 8) : 11%|█ | 2.49GB / 22.2GB, 244MB/s New Data Upload : 47%|████▋ | 848MB / 1.81GB, 82.9MB/s  ...eckpoints/model_070000.pt: 6%|▌ | 143MB / 2.35GB  ...nts/optim_070000_rank1.pt: 15%|█▍ | 362MB / 2.49GB  ...nts/optim_070000_rank2.pt: 14%|█▍ | 359MB / 2.49GB  ...nts/optim_070000_rank3.pt: 15%|█▍ | 365MB / 2.49GB  ...nts/optim_070000_rank4.pt: 15%|█▍ | 366MB / 2.49GB  ...nts/optim_070000_rank5.pt: 15%|█▍ | 362MB / 2.49GB  ...nts/optim_070000_rank6.pt: 15%|█▍ | 366MB / 2.49GB  ...nts/optim_070000_rank7.pt: 15%|█▍ | 365MB / 2.49GB  Processing Files (0 / 8) : 12%|█▏ | 2.69GB / 22.2GB, 263MB/s New Data Upload : 49%|████▊ | 881MB / 1.81GB, 85.9MB/s  ...eckpoints/model_070000.pt: 6%|▋ | 151MB / 2.35GB  ...nts/optim_070000_rank1.pt: 15%|█▌ | 385MB / 2.49GB  ...nts/optim_070000_rank2.pt: 15%|█▌ | 384MB / 2.49GB  ...nts/optim_070000_rank3.pt: 16%|█▌ | 388MB / 2.49GB  ...nts/optim_070000_rank4.pt: 16%|█▌ | 389MB / 2.49GB  ...nts/optim_070000_rank5.pt: 15%|█▌ | 385MB / 2.49GB  ...nts/optim_070000_rank6.pt: 16%|█▌ | 389MB / 2.49GB  ...nts/optim_070000_rank7.pt: 16%|█▌ | 388MB / 2.49GB  Processing Files (0 / 8) : 13%|█▎ | 2.86GB / 22.2GB, 279MB/s New Data Upload : 49%|████▊ | 915MB / 1.88GB, 88.9MB/s  ...eckpoints/model_070000.pt: 7%|▋ | 155MB / 2.35GB  ...nts/optim_070000_rank1.pt: 16%|█▋ | 406MB / 2.49GB  ...nts/optim_070000_rank2.pt: 16%|█▋ | 405MB / 2.49GB  ...nts/optim_070000_rank3.pt: 16%|█▋ | 409MB / 2.49GB  ...nts/optim_070000_rank4.pt: 16%|█▋ | 410MB / 2.49GB  ...nts/optim_070000_rank5.pt: 16%|█▋ | 406MB / 2.49GB  ...nts/optim_070000_rank6.pt: 17%|█▋ | 410MB / 2.49GB  ...nts/optim_070000_rank7.pt: 16%|█▋ | 410MB / 2.49GB  Processing Files (0 / 8) : 14%|█▎ | 3.01GB / 22.2GB, 294MB/s New Data Upload : 48%|████▊ | 942MB / 1.94GB, 91.1MB/s  ...eckpoints/model_070000.pt: 7%|▋ | 160MB / 2.35GB  ...nts/optim_070000_rank1.pt: 17%|█▋ | 425MB / 2.49GB  ...nts/optim_070000_rank2.pt: 17%|█▋ | 430MB / 2.49GB  ...nts/optim_070000_rank3.pt: 17%|█▋ | 429MB / 2.49GB  ...nts/optim_070000_rank4.pt: 17%|█▋ | 432MB / 2.49GB  ...nts/optim_070000_rank5.pt: 17%|█▋ | 427MB / 2.49GB  ...nts/optim_070000_rank6.pt: 17%|█▋ | 430MB / 2.49GB  ...nts/optim_070000_rank7.pt: 17%|█▋ | 429MB / 2.49GB  Processing Files (0 / 8) : 14%|█▍ | 3.16GB / 22.2GB, 308MB/s New Data Upload : 50%|█████ | 977MB / 1.94GB, 94.3MB/s  ...eckpoints/model_070000.pt: 7%|▋ | 172MB / 2.35GB  ...nts/optim_070000_rank1.pt: 18%|█▊ | 439MB / 2.49GB  ...nts/optim_070000_rank2.pt: 18%|█▊ | 447MB / 2.49GB  ...nts/optim_070000_rank3.pt: 18%|█▊ | 444MB / 2.49GB  ...nts/optim_070000_rank4.pt: 18%|█▊ | 450MB / 2.49GB  ...nts/optim_070000_rank5.pt: 18%|█▊ | 444MB / 2.49GB  ...nts/optim_070000_rank6.pt: 18%|█▊ | 445MB / 2.49GB  ...nts/optim_070000_rank7.pt: 18%|█▊ | 445MB / 2.49GB  Processing Files (0 / 8) : 15%|█▍ | 3.29GB / 22.2GB, 320MB/s New Data Upload : 49%|████▊ | 1.01GB / 2.08GB, 97.2MB/s  ...eckpoints/model_070000.pt: 8%|▊ | 179MB / 2.35GB  ...nts/optim_070000_rank1.pt: 19%|█▊ | 462MB / 2.49GB  ...nts/optim_070000_rank2.pt: 19%|█▉ | 471MB / 2.49GB  ...nts/optim_070000_rank3.pt: 19%|█▉ | 467MB / 2.49GB  ...nts/optim_070000_rank4.pt: 19%|█▉ | 476MB / 2.49GB  ...nts/optim_070000_rank5.pt: 19%|█▉ | 468MB / 2.49GB  ...nts/optim_070000_rank6.pt: 19%|█▉ | 468MB / 2.49GB  ...nts/optim_070000_rank7.pt: 19%|█▉ | 468MB / 2.49GB  Processing Files (0 / 8) : 16%|█▌ | 3.46GB / 22.2GB, 337MB/s New Data Upload : 47%|████▋ | 1.05GB / 2.21GB, 100MB/s  ...eckpoints/model_070000.pt: 8%|▊ | 187MB / 2.35GB  ...nts/optim_070000_rank1.pt: 19%|█▉ | 480MB / 2.49GB  ...nts/optim_070000_rank2.pt: 20%|█▉ | 491MB / 2.49GB  ...nts/optim_070000_rank3.pt: 20%|█▉ | 486MB / 2.49GB  ...nts/optim_070000_rank4.pt: 20%|█▉ | 494MB / 2.49GB  ...nts/optim_070000_rank5.pt: 20%|█▉ | 487MB / 2.49GB  ...nts/optim_070000_rank6.pt: 20%|█▉ | 486MB / 2.49GB  ...nts/optim_070000_rank7.pt: 20%|█▉ | 486MB / 2.49GB  Processing Files (0 / 8) : 16%|█▌ | 3.60GB / 22.2GB, 350MB/s New Data Upload : 49%|████▉ | 1.08GB / 2.21GB, 103MB/s  ...eckpoints/model_070000.pt: 8%|▊ | 191MB / 2.35GB  ...nts/optim_070000_rank1.pt: 20%|██ | 504MB / 2.49GB  ...nts/optim_070000_rank2.pt: 21%|██ | 517MB / 2.49GB  ...nts/optim_070000_rank3.pt: 21%|██ | 514MB / 2.49GB  ...nts/optim_070000_rank4.pt: 21%|██ | 521MB / 2.49GB  ...nts/optim_070000_rank5.pt: 21%|██ | 512MB / 2.49GB  ...nts/optim_070000_rank6.pt: 21%|██ | 511MB / 2.49GB  ...nts/optim_070000_rank7.pt: 21%|██ | 513MB / 2.49GB  Processing Files (0 / 8) : 17%|█▋ | 3.78GB / 22.2GB, 367MB/s New Data Upload : 49%|████▉ | 1.12GB / 2.28GB, 106MB/s  ...eckpoints/model_070000.pt: 9%|▊ | 200MB / 2.35GB  ...nts/optim_070000_rank1.pt: 21%|██▏ | 531MB / 2.49GB  ...nts/optim_070000_rank2.pt: 22%|██▏ | 545MB / 2.49GB  ...nts/optim_070000_rank3.pt: 22%|██▏ | 544MB / 2.49GB  ...nts/optim_070000_rank4.pt: 22%|██▏ | 551MB / 2.49GB  ...nts/optim_070000_rank5.pt: 22%|██▏ | 541MB / 2.49GB  ...nts/optim_070000_rank6.pt: 22%|██▏ | 540MB / 2.49GB  ...nts/optim_070000_rank7.pt: 22%|██▏ | 543MB / 2.49GB  Processing Files (0 / 8) : 18%|█▊ | 4.00GB / 22.2GB, 388MB/s New Data Upload : 51%|█████ | 1.17GB / 2.28GB, 111MB/s  ...eckpoints/model_070000.pt: 9%|▉ | 206MB / 2.35GB  ...nts/optim_070000_rank1.pt: 22%|██▏ | 553MB / 2.49GB  ...nts/optim_070000_rank2.pt: 23%|██▎ | 569MB / 2.49GB  ...nts/optim_070000_rank3.pt: 23%|██▎ | 569MB / 2.49GB  ...nts/optim_070000_rank4.pt: 23%|██▎ | 574MB / 2.49GB  ...nts/optim_070000_rank5.pt: 23%|██▎ | 564MB / 2.49GB  ...nts/optim_070000_rank6.pt: 23%|██▎ | 564MB / 2.49GB  ...nts/optim_070000_rank7.pt: 23%|██▎ | 567MB / 2.49GB  Processing Files (0 / 8) : 19%|█▊ | 4.17GB / 22.2GB, 404MB/s New Data Upload : 50%|████▉ | 1.20GB / 2.41GB, 114MB/s  ...eckpoints/model_070000.pt: 9%|▉ | 214MB / 2.35GB  ...nts/optim_070000_rank1.pt: 23%|██▎ | 567MB / 2.49GB  ...nts/optim_070000_rank2.pt: 24%|██▎ | 586MB / 2.49GB  ...nts/optim_070000_rank3.pt: 24%|██▎ | 587MB / 2.49GB  ...nts/optim_070000_rank4.pt: 24%|██▎ | 589MB / 2.49GB  ...nts/optim_070000_rank5.pt: 23%|██▎ | 578MB / 2.49GB  ...nts/optim_070000_rank6.pt: 23%|██▎ | 580MB / 2.49GB  ...nts/optim_070000_rank7.pt: 24%|██▎ | 584MB / 2.49GB  Processing Files (0 / 8) : 19%|█▉ | 4.29GB / 22.2GB, 415MB/s New Data Upload : 51%|█████▏ | 1.24GB / 2.41GB, 117MB/s  ...eckpoints/model_070000.pt: 10%|▉ | 223MB / 2.35GB  ...nts/optim_070000_rank1.pt: 24%|██▎ | 590MB / 2.49GB  ...nts/optim_070000_rank2.pt: 25%|██▍ | 610MB / 2.49GB  ...nts/optim_070000_rank3.pt: 25%|██▍ | 613MB / 2.49GB  ...nts/optim_070000_rank4.pt: 25%|██▍ | 614MB / 2.49GB  ...nts/optim_070000_rank5.pt: 24%|██▍ | 602MB / 2.49GB  ...nts/optim_070000_rank6.pt: 24%|██▍ | 606MB / 2.49GB  ...nts/optim_070000_rank7.pt: 25%|██▍ | 611MB / 2.49GB  Processing Files (0 / 8) : 20%|██ | 4.47GB / 22.2GB, 433MB/s New Data Upload : 53%|█████▎ | 1.28GB / 2.41GB, 121MB/s  ...eckpoints/model_070000.pt: 10%|▉ | 233MB / 2.35GB  ...nts/optim_070000_rank1.pt: 25%|██▍ | 613MB / 2.49GB  ...nts/optim_070000_rank2.pt: 26%|██▌ | 635MB / 2.49GB  ...nts/optim_070000_rank3.pt: 25%|██▌ | 634MB / 2.49GB  ...nts/optim_070000_rank4.pt: 26%|██▌ | 637MB / 2.49GB  ...nts/optim_070000_rank5.pt: 25%|██▌ | 626MB / 2.49GB  ...nts/optim_070000_rank6.pt: 25%|██▌ | 630MB / 2.49GB  ...nts/optim_070000_rank7.pt: 25%|██▌ | 633MB / 2.49GB  Processing Files (0 / 8) : 21%|██ | 4.64GB / 22.2GB, 449MB/s New Data Upload : 55%|█████▍ | 1.33GB / 2.41GB, 124MB/s  ...eckpoints/model_070000.pt: 10%|█ | 244MB / 2.35GB  ...nts/optim_070000_rank1.pt: 26%|██▌ | 636MB / 2.49GB  ...nts/optim_070000_rank2.pt: 26%|██▋ | 658MB / 2.49GB  ...nts/optim_070000_rank3.pt: 27%|██▋ | 660MB / 2.49GB  ...nts/optim_070000_rank4.pt: 27%|██▋ | 662MB / 2.49GB  ...nts/optim_070000_rank5.pt: 26%|██▌ | 649MB / 2.49GB  ...nts/optim_070000_rank6.pt: 26%|██▋ | 656MB / 2.49GB  ...nts/optim_070000_rank7.pt: 27%|██▋ | 660MB / 2.49GB  Processing Files (0 / 8) : 22%|██▏ | 4.82GB / 22.2GB, 467MB/s New Data Upload : 55%|█████▌ | 1.37GB / 2.48GB, 128MB/s  ...eckpoints/model_070000.pt: 11%|█ | 251MB / 2.35GB  ...nts/optim_070000_rank1.pt: 26%|██▋ | 654MB / 2.49GB  ...nts/optim_070000_rank2.pt: 27%|██▋ | 679MB / 2.49GB  ...nts/optim_070000_rank3.pt: 27%|██▋ | 681MB / 2.49GB  ...nts/optim_070000_rank4.pt: 27%|██▋ | 681MB / 2.49GB  ...nts/optim_070000_rank5.pt: 27%|██▋ | 669MB / 2.49GB  ...nts/optim_070000_rank6.pt: 27%|██▋ | 675MB / 2.49GB  ...nts/optim_070000_rank7.pt: 27%|██▋ | 682MB / 2.49GB  Processing Files (0 / 8) : 22%|██▏ | 4.97GB / 22.2GB, 481MB/s New Data Upload : 57%|█████▋ | 1.41GB / 2.48GB, 132MB/s  ...eckpoints/model_070000.pt: 11%|█ | 258MB / 2.35GB  ...nts/optim_070000_rank1.pt: 27%|██▋ | 677MB / 2.49GB  ...nts/optim_070000_rank2.pt: 28%|██▊ | 702MB / 2.49GB  ...nts/optim_070000_rank3.pt: 28%|██▊ | 707MB / 2.49GB  ...nts/optim_070000_rank4.pt: 28%|██▊ | 706MB / 2.49GB  ...nts/optim_070000_rank5.pt: 28%|██▊ | 691MB / 2.49GB  ...nts/optim_070000_rank6.pt: 28%|██▊ | 698MB / 2.49GB  ...nts/optim_070000_rank7.pt: 28%|██▊ | 707MB / 2.49GB  Processing Files (0 / 8) : 23%|██▎ | 5.15GB / 22.2GB, 498MB/s New Data Upload : 59%|█████▊ | 1.46GB / 2.48GB, 136MB/s  ...eckpoints/model_070000.pt: 11%|█▏ | 266MB / 2.35GB  ...nts/optim_070000_rank1.pt: 28%|██▊ | 698MB / 2.49GB  ...nts/optim_070000_rank2.pt: 29%|██▉ | 726MB / 2.49GB  ...nts/optim_070000_rank3.pt: 29%|██▉ | 729MB / 2.49GB  ...nts/optim_070000_rank4.pt: 29%|██▉ | 727MB / 2.49GB  ...nts/optim_070000_rank5.pt: 29%|██▊ | 713MB / 2.49GB  ...nts/optim_070000_rank6.pt: 29%|██▉ | 718MB / 2.49GB  ...nts/optim_070000_rank7.pt: 29%|██▉ | 731MB / 2.49GB  Processing Files (0 / 8) : 24%|██▍ | 5.31GB / 22.2GB, 514MB/s New Data Upload : 59%|█████▉ | 1.50GB / 2.55GB, 141MB/s  ...eckpoints/model_070000.pt: 12%|█▏ | 273MB / 2.35GB  ...nts/optim_070000_rank1.pt: 29%|██▉ | 715MB / 2.49GB  ...nts/optim_070000_rank2.pt: 30%|███ | 747MB / 2.49GB  ...nts/optim_070000_rank3.pt: 30%|███ | 749MB / 2.49GB  ...nts/optim_070000_rank4.pt: 30%|██▉ | 745MB / 2.49GB  ...nts/optim_070000_rank5.pt: 29%|██▉ | 731MB / 2.49GB  ...nts/optim_070000_rank6.pt: 30%|██▉ | 738MB / 2.49GB  ...nts/optim_070000_rank7.pt: 30%|███ | 750MB / 2.49GB  Processing Files (0 / 8) : 25%|██▍ | 5.45GB / 22.2GB, 527MB/s New Data Upload : 59%|█████▉ | 1.54GB / 2.62GB, 144MB/s  ...eckpoints/model_070000.pt: 12%|█▏ | 281MB / 2.35GB  ...nts/optim_070000_rank1.pt: 30%|██▉ | 734MB / 2.49GB  ...nts/optim_070000_rank2.pt: 31%|███ | 768MB / 2.49GB  ...nts/optim_070000_rank3.pt: 31%|███ | 772MB / 2.49GB  ...nts/optim_070000_rank4.pt: 31%|███ | 765MB / 2.49GB  ...nts/optim_070000_rank5.pt: 30%|███ | 750MB / 2.49GB  ...nts/optim_070000_rank6.pt: 31%|███ | 759MB / 2.49GB  ...nts/optim_070000_rank7.pt: 31%|███ | 773MB / 2.49GB  Processing Files (0 / 8) : 25%|██▌ | 5.60GB / 22.2GB, 542MB/s New Data Upload : 60%|██████ | 1.58GB / 2.62GB, 149MB/s  ...eckpoints/model_070000.pt: 12%|█▏ | 286MB / 2.35GB  ...nts/optim_070000_rank1.pt: 30%|███ | 757MB / 2.49GB  ...nts/optim_070000_rank2.pt: 32%|███▏ | 795MB / 2.49GB  ...nts/optim_070000_rank3.pt: 32%|███▏ | 798MB / 2.49GB  ...nts/optim_070000_rank4.pt: 32%|███▏ | 788MB / 2.49GB  ...nts/optim_070000_rank5.pt: 31%|███ | 773MB / 2.49GB  ...nts/optim_070000_rank6.pt: 31%|███▏ | 782MB / 2.49GB  ...nts/optim_070000_rank7.pt: 32%|███▏ | 799MB / 2.49GB  Processing Files (0 / 8) : 26%|██▌ | 5.78GB / 22.2GB, 559MB/s New Data Upload : 61%|██████ | 1.63GB / 2.68GB, 153MB/s  ...eckpoints/model_070000.pt: 12%|█▏ | 290MB / 2.35GB  ...nts/optim_070000_rank1.pt: 31%|███▏ | 781MB / 2.49GB  ...nts/optim_070000_rank2.pt: 33%|███▎ | 819MB / 2.49GB  ...nts/optim_070000_rank3.pt: 33%|███▎ | 821MB / 2.49GB  ...nts/optim_070000_rank4.pt: 33%|███▎ | 814MB / 2.49GB  ...nts/optim_070000_rank5.pt: 32%|███▏ | 797MB / 2.49GB  ...nts/optim_070000_rank6.pt: 32%|███▏ | 807MB / 2.49GB  ...nts/optim_070000_rank7.pt: 33%|███▎ | 825MB / 2.49GB  Processing Files (0 / 8) : 27%|██▋ | 5.95GB / 22.2GB, 576MB/s New Data Upload : 63%|██████▎ | 1.68GB / 2.68GB, 158MB/s  ...eckpoints/model_070000.pt: 13%|█▎ | 299MB / 2.35GB  ...nts/optim_070000_rank1.pt: 32%|███▏ | 800MB / 2.49GB  ...nts/optim_070000_rank2.pt: 34%|███▍ | 841MB / 2.49GB  ...nts/optim_070000_rank3.pt: 34%|███▍ | 843MB / 2.49GB  ...nts/optim_070000_rank4.pt: 34%|███▎ | 835MB / 2.49GB  ...nts/optim_070000_rank5.pt: 33%|███▎ | 815MB / 2.49GB  ...nts/optim_070000_rank6.pt: 33%|███▎ | 826MB / 2.49GB  ...nts/optim_070000_rank7.pt: 34%|███▍ | 846MB / 2.49GB  Processing Files (0 / 8) : 27%|██▋ | 6.10GB / 22.2GB, 591MB/s New Data Upload : 63%|██████▎ | 1.72GB / 2.75GB, 162MB/s  ...eckpoints/model_070000.pt: 13%|█▎ | 306MB / 2.35GB  ...nts/optim_070000_rank1.pt: 33%|███▎ | 821MB / 2.49GB  ...nts/optim_070000_rank2.pt: 35%|███▍ | 862MB / 2.49GB  ...nts/optim_070000_rank3.pt: 35%|███▍ | 865MB / 2.49GB  ...nts/optim_070000_rank4.pt: 34%|███▍ | 857MB / 2.49GB  ...nts/optim_070000_rank5.pt: 34%|███▎ | 836MB / 2.49GB  ...nts/optim_070000_rank6.pt: 34%|███▍ | 848MB / 2.49GB  ...nts/optim_070000_rank7.pt: 35%|███▍ | 868MB / 2.49GB  Processing Files (0 / 8) : 28%|██▊ | 6.26GB / 22.2GB, 606MB/s New Data Upload : 63%|██████▎ | 1.77GB / 2.82GB, 167MB/s  ...eckpoints/model_070000.pt: 13%|█▎ | 311MB / 2.35GB  ...nts/optim_070000_rank1.pt: 34%|███▍ | 847MB / 2.49GB  ...nts/optim_070000_rank2.pt: 36%|███▌ | 887MB / 2.49GB  ...nts/optim_070000_rank3.pt: 36%|███▌ | 887MB / 2.49GB  ...nts/optim_070000_rank4.pt: 35%|███▌ | 880MB / 2.49GB  ...nts/optim_070000_rank5.pt: 35%|███▍ | 862MB / 2.49GB  ...nts/optim_070000_rank6.pt: 35%|███▌ | 875MB / 2.49GB  ...nts/optim_070000_rank7.pt: 36%|███▌ | 895MB / 2.49GB  Processing Files (0 / 8) : 29%|██▉ | 6.44GB / 22.2GB, 622MB/s New Data Upload : 63%|██████▎ | 1.82GB / 2.88GB, 171MB/s  ...eckpoints/model_070000.pt: 13%|█▎ | 315MB / 2.35GB  ...nts/optim_070000_rank1.pt: 35%|███▍ | 868MB / 2.49GB  ...nts/optim_070000_rank2.pt: 36%|███▋ | 907MB / 2.49GB  ...nts/optim_070000_rank3.pt: 36%|███▋ | 906MB / 2.49GB  ...nts/optim_070000_rank4.pt: 36%|███▋ | 905MB / 2.49GB  ...nts/optim_070000_rank5.pt: 36%|███▌ | 883MB / 2.49GB  ...nts/optim_070000_rank6.pt: 36%|███▌ | 899MB / 2.49GB  ...nts/optim_070000_rank7.pt: 37%|███▋ | 914MB / 2.49GB  Processing Files (0 / 8) : 30%|██▉ | 6.60GB / 22.2GB, 634MB/s New Data Upload : 64%|██████▍ | 1.86GB / 2.88GB, 174MB/s  ...eckpoints/model_070000.pt: 14%|█▎ | 321MB / 2.35GB  ...nts/optim_070000_rank1.pt: 35%|███▌ | 881MB / 2.49GB  ...nts/optim_070000_rank2.pt: 37%|███▋ | 920MB / 2.49GB  ...nts/optim_070000_rank3.pt: 37%|███▋ | 917MB / 2.49GB  ...nts/optim_070000_rank4.pt: 37%|███▋ | 919MB / 2.49GB  ...nts/optim_070000_rank5.pt: 36%|███▌ | 896MB / 2.49GB  ...nts/optim_070000_rank6.pt: 37%|███▋ | 914MB / 2.49GB  ...nts/optim_070000_rank7.pt: 37%|███▋ | 926MB / 2.49GB  Processing Files (0 / 8) : 30%|███ | 6.69GB / 22.2GB, 639MB/s New Data Upload : 66%|██████▌ | 1.89GB / 2.88GB, 176MB/s  ...eckpoints/model_070000.pt: 14%|█▎ | 321MB / 2.35GB  ...nts/optim_070000_rank1.pt: 36%|███▌ | 895MB / 2.49GB  ...nts/optim_070000_rank2.pt: 38%|███▊ | 938MB / 2.49GB  ...nts/optim_070000_rank3.pt: 38%|███▊ | 933MB / 2.49GB  ...nts/optim_070000_rank4.pt: 38%|███▊ | 936MB / 2.49GB  ...nts/optim_070000_rank5.pt: 37%|███▋ | 909MB / 2.49GB  ...nts/optim_070000_rank6.pt: 37%|███▋ | 931MB / 2.49GB  ...nts/optim_070000_rank7.pt: 38%|███▊ | 940MB / 2.49GB  Processing Files (0 / 8) : 31%|███ | 6.80GB / 22.2GB, 647MB/s New Data Upload : 63%|██████▎ | 1.92GB / 3.02GB, 177MB/s  ...eckpoints/model_070000.pt: 14%|█▍ | 326MB / 2.35GB  ...nts/optim_070000_rank1.pt: 37%|███▋ | 915MB / 2.49GB  ...nts/optim_070000_rank2.pt: 38%|███▊ | 957MB / 2.49GB  ...nts/optim_070000_rank3.pt: 38%|███▊ | 949MB / 2.49GB  ...nts/optim_070000_rank4.pt: 38%|███▊ | 956MB / 2.49GB  ...nts/optim_070000_rank5.pt: 37%|███▋ | 930MB / 2.49GB  ...nts/optim_070000_rank6.pt: 38%|███▊ | 953MB / 2.49GB  ...nts/optim_070000_rank7.pt: 39%|███▊ | 959MB / 2.49GB  Processing Files (0 / 8) : 31%|███ | 6.95GB / 22.2GB, 656MB/s New Data Upload : 65%|██████▍ | 1.96GB / 3.02GB, 179MB/s  ...eckpoints/model_070000.pt: 14%|█▍ | 329MB / 2.35GB  ...nts/optim_070000_rank1.pt: 38%|███▊ | 942MB / 2.49GB  ...nts/optim_070000_rank2.pt: 40%|███▉ | 984MB / 2.49GB  ...nts/optim_070000_rank3.pt: 39%|███▉ | 974MB / 2.49GB  ...nts/optim_070000_rank4.pt: 39%|███▉ | 981MB / 2.49GB  ...nts/optim_070000_rank5.pt: 39%|███▊ | 957MB / 2.49GB  ...nts/optim_070000_rank6.pt: 39%|███▉ | 981MB / 2.49GB  ...nts/optim_070000_rank7.pt: 40%|███▉ | 985MB / 2.49GB  Processing Files (0 / 8) : 32%|███▏ | 7.13GB / 22.2GB, 669MB/s New Data Upload : 66%|██████▌ | 2.00GB / 3.02GB, 181MB/s  ...eckpoints/model_070000.pt: 14%|█▍ | 331MB / 2.35GB  ...nts/optim_070000_rank1.pt: 39%|███▊ | 960MB / 2.49GB  ...nts/optim_070000_rank2.pt: 40%|████ | 1.00GB / 2.49GB  ...nts/optim_070000_rank3.pt: 40%|███▉ | 990MB / 2.49GB  ...nts/optim_070000_rank4.pt: 40%|████ | 1.00GB / 2.49GB  ...nts/optim_070000_rank5.pt: 39%|███▉ | 975MB / 2.49GB  ...nts/optim_070000_rank6.pt: 40%|████ | 999MB / 2.49GB  ...nts/optim_070000_rank7.pt: 40%|████ | 1.00GB / 2.49GB  Processing Files (0 / 8) : 33%|███▎ | 7.26GB / 22.2GB, 676MB/s New Data Upload : 64%|██████▍ | 2.03GB / 3.15GB, 182MB/s  ...eckpoints/model_070000.pt: 14%|█▍ | 334MB / 2.35GB  ...nts/optim_070000_rank1.pt: 39%|███▉ | 978MB / 2.49GB  ...nts/optim_070000_rank2.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_070000_rank3.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_070000_rank4.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_070000_rank5.pt: 40%|███▉ | 992MB / 2.49GB  ...nts/optim_070000_rank6.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_070000_rank7.pt: 41%|████ | 1.02GB / 2.49GB  Processing Files (0 / 8) : 33%|███▎ | 7.39GB / 22.2GB, 682MB/s New Data Upload : 64%|██████▍ | 2.06GB / 3.22GB, 183MB/s  ...eckpoints/model_070000.pt: 14%|█▍ | 337MB / 2.35GB  ...nts/optim_070000_rank1.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_070000_rank2.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_070000_rank3.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_070000_rank4.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_070000_rank5.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_070000_rank6.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_070000_rank7.pt: 42%|████▏ | 1.05GB / 2.49GB  Processing Files (0 / 8) : 34%|███▍ | 7.60GB / 22.2GB, 699MB/s New Data Upload : 64%|██████▍ | 2.11GB / 3.29GB, 186MB/s  ...eckpoints/model_070000.pt: 14%|█▍ | 340MB / 2.35GB  ...nts/optim_070000_rank1.pt: 42%|████▏ | 1.03GB / 2.49GB  ...nts/optim_070000_rank2.pt: 43%|████▎ | 1.08GB / 2.49GB  ...nts/optim_070000_rank3.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_070000_rank4.pt: 43%|████▎ | 1.08GB / 2.49GB  ...nts/optim_070000_rank5.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_070000_rank6.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_070000_rank7.pt: 43%|████▎ | 1.07GB / 2.49GB  Processing Files (0 / 8) : 35%|███▍ | 7.77GB / 22.2GB, 710MB/s New Data Upload : 64%|██████▍ | 2.15GB / 3.35GB, 187MB/s  ...eckpoints/model_070000.pt: 15%|█▍ | 345MB / 2.35GB  ...nts/optim_070000_rank1.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_070000_rank2.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_070000_rank3.pt: 43%|████▎ | 1.08GB / 2.49GB  ...nts/optim_070000_rank4.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_070000_rank5.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_070000_rank6.pt: 44%|████▍ | 1.09GB / 2.49GB  ...nts/optim_070000_rank7.pt: 44%|████▍ | 1.09GB / 2.49GB  Processing Files (0 / 8) : 36%|███▌ | 7.93GB / 22.2GB, 719MB/s New Data Upload : 61%|██████▏ | 2.19GB / 3.55GB, 189MB/s  ...eckpoints/model_070000.pt: 15%|█▍ | 351MB / 2.35GB  ...nts/optim_070000_rank1.pt: 43%|████▎ | 1.08GB / 2.49GB  ...nts/optim_070000_rank2.pt: 45%|████▌ | 1.13GB / 2.49GB  ...nts/optim_070000_rank3.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_070000_rank4.pt: 45%|████▌ | 1.13GB / 2.49GB  ...nts/optim_070000_rank5.pt: 44%|████▍ | 1.09GB / 2.49GB  ...nts/optim_070000_rank6.pt: 45%|████▍ | 1.12GB / 2.49GB  ...nts/optim_070000_rank7.pt: 45%|████▍ | 1.12GB / 2.49GB  Processing Files (0 / 8) : 36%|███▋ | 8.11GB / 22.2GB, 732MB/s New Data Upload : 62%|██████▏ | 2.23GB / 3.62GB, 190MB/s  ...eckpoints/model_070000.pt: 15%|█▌ | 356MB / 2.35GB  ...nts/optim_070000_rank1.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_070000_rank2.pt: 46%|████▌ | 1.15GB / 2.49GB  ...nts/optim_070000_rank3.pt: 45%|████▌ | 1.13GB / 2.49GB  ...nts/optim_070000_rank4.pt: 46%|████▌ | 1.15GB / 2.49GB  ...nts/optim_070000_rank5.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_070000_rank6.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_070000_rank7.pt: 46%|████▌ | 1.14GB / 2.49GB  Processing Files (0 / 8) : 37%|███▋ | 8.27GB / 22.2GB, 743MB/s New Data Upload : 61%|██████▏ | 2.27GB / 3.69GB, 192MB/s  ...eckpoints/model_070000.pt: 15%|█▌ | 363MB / 2.35GB  ...nts/optim_070000_rank1.pt: 45%|████▌ | 1.13GB / 2.49GB  ...nts/optim_070000_rank2.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_070000_rank3.pt: 46%|████▌ | 1.15GB / 2.49GB  ...nts/optim_070000_rank4.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_070000_rank5.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_070000_rank6.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_070000_rank7.pt: 47%|████▋ | 1.16GB / 2.49GB  Processing Files (0 / 8) : 38%|███▊ | 8.44GB / 22.2GB, 755MB/s New Data Upload : 61%|██████▏ | 2.31GB / 3.76GB, 193MB/s  ...eckpoints/model_070000.pt: 16%|█▌ | 371MB / 2.35GB  ...nts/optim_070000_rank1.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_070000_rank2.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_070000_rank3.pt: 47%|████▋ | 1.18GB / 2.49GB  ...nts/optim_070000_rank4.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_070000_rank5.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_070000_rank6.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_070000_rank7.pt: 48%|████▊ | 1.19GB / 2.49GB  Processing Files (0 / 8) : 39%|███▉ | 8.65GB / 22.2GB, 771MB/s New Data Upload : 63%|██████▎ | 2.35GB / 3.76GB, 195MB/s  ...eckpoints/model_070000.pt: 16%|█▌ | 378MB / 2.35GB  ...nts/optim_070000_rank1.pt: 48%|████▊ | 1.18GB / 2.49GB  ...nts/optim_070000_rank2.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_070000_rank3.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_070000_rank4.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_070000_rank5.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_070000_rank6.pt: 49%|████▉ | 1.21GB / 2.49GB  ...nts/optim_070000_rank7.pt: 49%|████▉ | 1.22GB / 2.49GB  Processing Files (0 / 8) : 40%|███▉ | 8.83GB / 22.2GB, 781MB/s New Data Upload : 63%|██████▎ | 2.39GB / 3.82GB, 196MB/s  ...eckpoints/model_070000.pt: 16%|█▋ | 386MB / 2.35GB  ...nts/optim_070000_rank1.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_070000_rank2.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_070000_rank3.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_070000_rank4.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_070000_rank5.pt: 49%|████▉ | 1.21GB / 2.49GB  ...nts/optim_070000_rank6.pt: 50%|████▉ | 1.23GB / 2.49GB  ...nts/optim_070000_rank7.pt: 50%|████▉ | 1.24GB / 2.49GB  Processing Files (0 / 8) : 40%|████ | 8.97GB / 22.2GB, 788MB/s New Data Upload : 63%|██████▎ | 2.43GB / 3.82GB, 196MB/s  ...eckpoints/model_070000.pt: 17%|█▋ | 390MB / 2.35GB  ...nts/optim_070000_rank1.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_070000_rank2.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_070000_rank3.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_070000_rank4.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_070000_rank5.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_070000_rank6.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_070000_rank7.pt: 51%|█████ | 1.27GB / 2.49GB  Processing Files (0 / 8) : 41%|████▏ | 9.19GB / 22.2GB, 803MB/s New Data Upload : 64%|██████▎ | 2.47GB / 3.89GB, 197MB/s  ...eckpoints/model_070000.pt: 17%|█▋ | 399MB / 2.35GB  ...nts/optim_070000_rank1.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_070000_rank2.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_070000_rank3.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_070000_rank4.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_070000_rank5.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_070000_rank6.pt: 52%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_070000_rank7.pt: 52%|█████▏ | 1.29GB / 2.49GB  Processing Files (0 / 8) : 42%|████▏ | 9.33GB / 22.2GB, 807MB/s New Data Upload : 65%|██████▍ | 2.51GB / 3.89GB, 197MB/s  ...eckpoints/model_070000.pt: 17%|█▋ | 407MB / 2.35GB  ...nts/optim_070000_rank1.pt: 52%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_070000_rank2.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_070000_rank3.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_070000_rank4.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_070000_rank5.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_070000_rank6.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_070000_rank7.pt: 53%|█████▎ | 1.31GB / 2.49GB  Processing Files (0 / 8) : 43%|████▎ | 9.51GB / 22.2GB, 815MB/s New Data Upload : 66%|██████▌ | 2.55GB / 3.89GB, 197MB/s  ...eckpoints/model_070000.pt: 18%|█▊ | 412MB / 2.35GB  ...nts/optim_070000_rank1.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_070000_rank2.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_070000_rank3.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_070000_rank4.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_070000_rank5.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_070000_rank6.pt: 54%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_070000_rank7.pt: 54%|█████▍ | 1.34GB / 2.49GB  Processing Files (0 / 8) : 44%|████▎ | 9.71GB / 22.2GB, 827MB/s New Data Upload : 65%|██████▍ | 2.60GB / 4.02GB, 198MB/s  ...eckpoints/model_070000.pt: 18%|█▊ | 420MB / 2.35GB  ...nts/optim_070000_rank1.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_070000_rank2.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_070000_rank3.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_070000_rank4.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_070000_rank5.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_070000_rank6.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_070000_rank7.pt: 55%|█████▌ | 1.37GB / 2.49GB  Processing Files (0 / 8) : 45%|████▍ | 9.94GB / 22.2GB, 835MB/s New Data Upload : 66%|██████▌ | 2.65GB / 4.02GB, 199MB/s  ...eckpoints/model_070000.pt: 18%|█▊ | 428MB / 2.35GB  ...nts/optim_070000_rank1.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_070000_rank2.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_070000_rank3.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_070000_rank4.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_070000_rank5.pt: 56%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_070000_rank6.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_070000_rank7.pt: 56%|█████▌ | 1.40GB / 2.49GB  Processing Files (0 / 8) : 46%|████▌ | 10.1GB / 22.2GB, 844MB/s New Data Upload : 65%|██████▍ | 2.70GB / 4.16GB, 201MB/s  ...eckpoints/model_070000.pt: 19%|█▊ | 438MB / 2.35GB  ...nts/optim_070000_rank1.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_070000_rank2.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_070000_rank3.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...nts/optim_070000_rank4.pt: 58%|█████▊ | 1.43GB / 2.49GB  ...nts/optim_070000_rank5.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...nts/optim_070000_rank6.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_070000_rank7.pt: 57%|█████▋ | 1.42GB / 2.49GB  Processing Files (0 / 8) : 47%|████▋ | 10.4GB / 22.2GB, 848MB/s New Data Upload : 65%|██████▌ | 2.75GB / 4.23GB, 202MB/s  ...eckpoints/model_070000.pt: 19%|█▉ | 446MB / 2.35GB  ...nts/optim_070000_rank1.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_070000_rank2.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_070000_rank3.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_070000_rank4.pt: 59%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_070000_rank5.pt: 58%|█████▊ | 1.43GB / 2.49GB  ...nts/optim_070000_rank6.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_070000_rank7.pt: 58%|█████▊ | 1.45GB / 2.49GB  Processing Files (0 / 8) : 47%|████▋ | 10.5GB / 22.2GB, 843MB/s New Data Upload : 62%|██████▏ | 2.79GB / 4.49GB, 202MB/s  ...eckpoints/model_070000.pt: 20%|█▉ | 458MB / 2.35GB  ...nts/optim_070000_rank1.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_070000_rank2.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_070000_rank3.pt: 59%|█████▉ | 1.46GB / 2.49GB  ...nts/optim_070000_rank4.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_070000_rank5.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_070000_rank6.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_070000_rank7.pt: 60%|█████▉ | 1.48GB / 2.49GB  Processing Files (0 / 8) : 48%|████▊ | 10.8GB / 22.2GB, 847MB/s New Data Upload : 62%|██████▏ | 2.85GB / 4.56GB, 203MB/s  ...eckpoints/model_070000.pt: 20%|█▉ | 467MB / 2.35GB  ...nts/optim_070000_rank1.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_070000_rank3.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_070000_rank5.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_070000_rank7.pt: 61%|██████ | 1.51GB / 2.49GB  Processing Files (0 / 8) : 49%|████▉ | 11.0GB / 22.2GB, 841MB/s New Data Upload : 63%|██████▎ | 2.90GB / 4.56GB, 203MB/s  ...eckpoints/model_070000.pt: 20%|██ | 475MB / 2.35GB  ...nts/optim_070000_rank1.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_070000_rank2.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_070000_rank3.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_070000_rank4.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_070000_rank5.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_070000_rank6.pt: 61%|██████▏ | 1.52GB / 2.49GB  ...nts/optim_070000_rank7.pt: 62%|██████▏ | 1.53GB / 2.49GB  Processing Files (0 / 8) : 50%|█████ | 11.1GB / 22.2GB, 847MB/s New Data Upload : 64%|██████▍ | 2.94GB / 4.56GB, 205MB/s  ...eckpoints/model_070000.pt: 20%|██ | 479MB / 2.35GB  ...nts/optim_070000_rank1.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_070000_rank2.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_070000_rank3.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_070000_rank4.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_070000_rank5.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_070000_rank6.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_070000_rank7.pt: 63%|██████▎ | 1.55GB / 2.49GB  Processing Files (0 / 8) : 51%|█████ | 11.3GB / 22.2GB, 845MB/s New Data Upload : 64%|██████▍ | 2.97GB / 4.63GB, 205MB/s  ...eckpoints/model_070000.pt: 21%|██ | 484MB / 2.35GB  ...nts/optim_070000_rank1.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_070000_rank2.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_070000_rank3.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_070000_rank4.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_070000_rank5.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_070000_rank6.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_070000_rank7.pt: 64%|██████▎ | 1.58GB / 2.49GB  Processing Files (0 / 8) : 52%|█████▏ | 11.5GB / 22.2GB, 848MB/s New Data Upload : 65%|██████▌ | 3.02GB / 4.63GB, 207MB/s  ...eckpoints/model_070000.pt: 21%|██ | 492MB / 2.35GB  ...nts/optim_070000_rank1.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_070000_rank2.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_070000_rank3.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_070000_rank4.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_070000_rank5.pt: 64%|██████▍ | 1.58GB / 2.49GB  ...nts/optim_070000_rank6.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_070000_rank7.pt: 64%|██████▍ | 1.60GB / 2.49GB  Processing Files (0 / 8) : 52%|█████▏ | 11.7GB / 22.2GB, 849MB/s New Data Upload : 66%|██████▌ | 3.06GB / 4.63GB, 208MB/s  ...eckpoints/model_070000.pt: 21%|██ | 494MB / 2.35GB  ...nts/optim_070000_rank1.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_070000_rank2.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_070000_rank3.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_070000_rank4.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_070000_rank5.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_070000_rank6.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_070000_rank7.pt: 65%|██████▌ | 1.63GB / 2.49GB  Processing Files (0 / 8) : 53%|█████▎ | 11.8GB / 22.2GB, 852MB/s New Data Upload : 66%|██████▋ | 3.10GB / 4.67GB, 208MB/s  ...nts/optim_070000_rank0.pt: 5%|▍ | 112MB / 2.49GB  ...eckpoints/model_070000.pt: 21%|██▏ | 500MB / 2.35GB  ...nts/optim_070000_rank1.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_070000_rank2.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_070000_rank3.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_070000_rank4.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_070000_rank5.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_070000_rank6.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_070000_rank7.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_070000_rank0.pt: 5%|▍ | 112MB / 2.49GB  Processing Files (0 / 9) : 54%|█████▍ | 12.1GB / 22.2GB, 866MB/s New Data Upload : 67%|██████▋ | 3.14GB / 4.67GB, 209MB/s  ...eckpoints/model_070000.pt: 21%|██▏ | 502MB / 2.35GB  ...nts/optim_070000_rank1.pt: 67%|██████▋ | 1.68GB / 2.49GB  ...nts/optim_070000_rank2.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_070000_rank3.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_070000_rank4.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_070000_rank5.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_070000_rank6.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_070000_rank7.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_070000_rank0.pt: 10%|▉ | 238MB / 2.49GB  Processing Files (0 / 9) : 56%|█████▌ | 12.5GB / 22.2GB, 883MB/s New Data Upload : 68%|██████▊ | 3.19GB / 4.67GB, 210MB/s  ...eckpoints/model_070000.pt: 22%|██▏ | 507MB / 2.35GB  ...nts/optim_070000_rank1.pt: 69%|██████▊ | 1.71GB / 2.49GB  ...nts/optim_070000_rank2.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_070000_rank3.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_070000_rank4.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_070000_rank5.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_070000_rank6.pt: 69%|██████▊ | 1.71GB / 2.49GB  ...nts/optim_070000_rank7.pt: 69%|██████▊ | 1.71GB / 2.49GB  ...nts/optim_070000_rank0.pt: 14%|█▍ | 359MB / 2.49GB  Processing Files (0 / 9) : 58%|█████▊ | 12.8GB / 22.2GB, 902MB/s New Data Upload : 69%|██████▉ | 3.24GB / 4.67GB, 211MB/s  ...eckpoints/model_070000.pt: 22%|██▏ | 510MB / 2.35GB  ...nts/optim_070000_rank1.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_070000_rank2.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_070000_rank3.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_070000_rank4.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_070000_rank5.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_070000_rank6.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_070000_rank7.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_070000_rank0.pt: 18%|█▊ | 440MB / 2.49GB  Processing Files (0 / 9) : 59%|█████▊ | 13.1GB / 22.2GB, 910MB/s New Data Upload : 70%|███████ | 3.28GB / 4.67GB, 212MB/s  ...eckpoints/model_070000.pt: 22%|██▏ | 515MB / 2.35GB  ...nts/optim_070000_rank1.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_070000_rank2.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_070000_rank3.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_070000_rank4.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_070000_rank5.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_070000_rank6.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_070000_rank7.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_070000_rank0.pt: 22%|██▏ | 542MB / 2.49GB  Processing Files (0 / 9) : 60%|██████ | 13.4GB / 22.2GB, 919MB/s New Data Upload : 71%|███████▏ | 3.33GB / 4.67GB, 212MB/s  ...eckpoints/model_070000.pt: 22%|██▏ | 518MB / 2.35GB  ...nts/optim_070000_rank1.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_070000_rank2.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_070000_rank3.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_070000_rank4.pt: 73%|███████▎ | 1.80GB / 2.49GB  ...nts/optim_070000_rank5.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_070000_rank6.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_070000_rank7.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_070000_rank0.pt: 29%|██▉ | 725MB / 2.49GB  Processing Files (0 / 9) : 62%|██████▏ | 13.8GB / 22.2GB, 944MB/s New Data Upload : 70%|███████ | 3.38GB / 4.81GB, 214MB/s  ...eckpoints/model_070000.pt: 22%|██▏ | 522MB / 2.35GB  ...nts/optim_070000_rank1.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_070000_rank2.pt: 74%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_070000_rank3.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_070000_rank4.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_070000_rank5.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_070000_rank6.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_070000_rank7.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_070000_rank0.pt: 35%|███▌ | 878MB / 2.49GB  Processing Files (0 / 9) : 63%|██████▎ | 14.1GB / 22.2GB, 958MB/s New Data Upload : 71%|███████ | 3.42GB / 4.81GB, 214MB/s  ...eckpoints/model_070000.pt: 23%|██▎ | 530MB / 2.35GB  ...nts/optim_070000_rank1.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_070000_rank2.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_070000_rank3.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_070000_rank4.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_070000_rank5.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_070000_rank6.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_070000_rank7.pt: 74%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_070000_rank0.pt: 39%|███▉ | 981MB / 2.49GB  Processing Files (0 / 9) : 65%|██████▍ | 14.4GB / 22.2GB, 970MB/s New Data Upload : 72%|███████▏ | 3.47GB / 4.81GB, 214MB/s  ...eckpoints/model_070000.pt: 23%|██▎ | 536MB / 2.35GB  ...nts/optim_070000_rank1.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_070000_rank2.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_070000_rank3.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_070000_rank4.pt: 75%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_070000_rank5.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_070000_rank6.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_070000_rank7.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_070000_rank0.pt: 43%|████▎ | 1.07GB / 2.49GB  Processing Files (0 / 9) : 66%|██████▌ | 14.6GB / 22.2GB, 980MB/s New Data Upload : 72%|███████▏ | 3.52GB / 4.87GB, 215MB/s  ...eckpoints/model_070000.pt: 23%|██▎ | 552MB / 2.35GB  ...nts/optim_070000_rank1.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_070000_rank2.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_070000_rank3.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_070000_rank4.pt: 77%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_070000_rank5.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_070000_rank6.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_070000_rank7.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_070000_rank0.pt: 46%|████▋ | 1.16GB / 2.49GB  Processing Files (0 / 9) : 67%|██████▋ | 14.9GB / 22.2GB, 989MB/s New Data Upload : 72%|███████▏ | 3.58GB / 4.94GB, 216MB/s  ...eckpoints/model_070000.pt: 24%|██▍ | 562MB / 2.35GB  ...nts/optim_070000_rank1.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_070000_rank2.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_070000_rank3.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_070000_rank4.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_070000_rank5.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_070000_rank6.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_070000_rank7.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_070000_rank0.pt: 51%|█████▏ | 1.28GB / 2.49GB  Processing Files (0 / 9) : 68%|██████▊ | 15.2GB / 22.2GB, 1.00GB/s New Data Upload : 73%|███████▎ | 3.62GB / 4.94GB, 217MB/s  ...eckpoints/model_070000.pt: 24%|██▍ | 572MB / 2.35GB  ...nts/optim_070000_rank1.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_070000_rank2.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_070000_rank3.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_070000_rank4.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_070000_rank5.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_070000_rank6.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_070000_rank7.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_070000_rank0.pt: 55%|█████▌ | 1.38GB / 2.49GB  Processing Files (0 / 9) : 69%|██████▉ | 15.4GB / 22.2GB, 1.01GB/s New Data Upload : 73%|███████▎ | 3.67GB / 5.01GB, 217MB/s  ...eckpoints/model_070000.pt: 25%|██▌ | 596MB / 2.35GB  ...nts/optim_070000_rank1.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_070000_rank2.pt: 79%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_070000_rank3.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_070000_rank4.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_070000_rank5.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_070000_rank6.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_070000_rank7.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_070000_rank0.pt: 59%|█████▉ | 1.47GB / 2.49GB  Processing Files (0 / 9) : 71%|███████ | 15.7GB / 22.2GB, 1.02GB/s New Data Upload : 75%|███████▍ | 3.73GB / 5.01GB, 219MB/s  ...eckpoints/model_070000.pt: 26%|██▌ | 608MB / 2.35GB  ...nts/optim_070000_rank1.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_070000_rank2.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_070000_rank3.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_070000_rank4.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_070000_rank5.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_070000_rank6.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_070000_rank7.pt: 79%|███████▊ | 1.96GB / 2.49GB  ...nts/optim_070000_rank0.pt: 63%|██████▎ | 1.57GB / 2.49GB  Processing Files (0 / 9) : 72%|███████▏ | 16.0GB / 22.2GB, 1.03GB/s New Data Upload : 74%|███████▍ | 3.78GB / 5.08GB, 219MB/s  ...eckpoints/model_070000.pt: 27%|██▋ | 627MB / 2.35GB  ...nts/optim_070000_rank1.pt: 80%|███████▉ | 1.99GB / 2.49GB  ...nts/optim_070000_rank2.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_070000_rank3.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_070000_rank4.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_070000_rank5.pt: 79%|███████▊ | 1.96GB / 2.49GB  ...nts/optim_070000_rank6.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_070000_rank7.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_070000_rank0.pt: 65%|██████▌ | 1.62GB / 2.49GB  Processing Files (0 / 9) : 73%|███████▎ | 16.1GB / 22.2GB, 1.03GB/s New Data Upload : 75%|███████▌ | 3.82GB / 5.08GB, 220MB/s  ...eckpoints/model_070000.pt: 28%|██▊ | 646MB / 2.35GB  ...nts/optim_070000_rank1.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_070000_rank2.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_070000_rank3.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_070000_rank4.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_070000_rank5.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_070000_rank6.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_070000_rank7.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_070000_rank0.pt: 68%|██████▊ | 1.69GB / 2.49GB  Processing Files (0 / 9) : 74%|███████▎ | 16.4GB / 22.2GB, 1.04GB/s New Data Upload : 75%|███████▌ | 3.88GB / 5.14GB, 220MB/s  ...eckpoints/model_070000.pt: 28%|██▊ | 669MB / 2.35GB  ...nts/optim_070000_rank1.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_070000_rank2.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_070000_rank3.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_070000_rank4.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_070000_rank5.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_070000_rank6.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_070000_rank7.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_070000_rank0.pt: 73%|███████▎ | 1.81GB / 2.49GB  Processing Files (0 / 9) : 75%|███████▍ | 16.6GB / 22.2GB, 1.05GB/s New Data Upload : 76%|███████▋ | 3.93GB / 5.14GB, 221MB/s  ...eckpoints/model_070000.pt: 30%|██▉ | 694MB / 2.35GB  ...nts/optim_070000_rank1.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_070000_rank2.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_070000_rank3.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_070000_rank4.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_070000_rank5.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_070000_rank6.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_070000_rank7.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_070000_rank0.pt: 76%|███████▌ | 1.88GB / 2.49GB  Processing Files (0 / 9) : 76%|███████▌ | 16.8GB / 22.2GB, 1.05GB/s New Data Upload : 76%|███████▋ | 3.98GB / 5.21GB, 221MB/s  ...eckpoints/model_070000.pt: 31%|███ | 720MB / 2.35GB  ...nts/optim_070000_rank1.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_070000_rank2.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_070000_rank3.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_070000_rank4.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_070000_rank5.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_070000_rank6.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_070000_rank7.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_070000_rank0.pt: 80%|████████ | 1.99GB / 2.49GB  Processing Files (0 / 9) : 77%|███████▋ | 17.1GB / 22.2GB, 1.06GB/s New Data Upload : 76%|███████▋ | 4.03GB / 5.28GB, 222MB/s  ...eckpoints/model_070000.pt: 32%|███▏ | 744MB / 2.35GB  ...nts/optim_070000_rank1.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_070000_rank2.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_070000_rank3.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_070000_rank4.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_070000_rank5.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_070000_rank6.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_070000_rank7.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_070000_rank0.pt: 82%|████████▏ | 2.03GB / 2.49GB  Processing Files (0 / 9) : 78%|███████▊ | 17.2GB / 22.2GB, 1.06GB/s New Data Upload : 77%|███████▋ | 4.08GB / 5.28GB, 222MB/s  ...eckpoints/model_070000.pt: 33%|███▎ | 772MB / 2.35GB  ...nts/optim_070000_rank1.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_070000_rank2.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_070000_rank3.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_070000_rank4.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_070000_rank5.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_070000_rank6.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_070000_rank7.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_070000_rank0.pt: 82%|████████▏ | 2.05GB / 2.49GB  Processing Files (0 / 9) : 78%|███████▊ | 17.4GB / 22.2GB, 1.06GB/s New Data Upload : 77%|███████▋ | 4.14GB / 5.34GB, 223MB/s  ...eckpoints/model_070000.pt: 34%|███▎ | 792MB / 2.35GB  ...nts/optim_070000_rank1.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_070000_rank2.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_070000_rank3.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_070000_rank4.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_070000_rank5.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_070000_rank6.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_070000_rank7.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_070000_rank0.pt: 83%|████████▎ | 2.06GB / 2.49GB  Processing Files (0 / 9) : 79%|███████▉ | 17.5GB / 22.2GB, 1.06GB/s New Data Upload : 78%|███████▊ | 4.17GB / 5.34GB, 224MB/s  ...eckpoints/model_070000.pt: 35%|███▍ | 821MB / 2.35GB  ...nts/optim_070000_rank1.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_070000_rank2.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_070000_rank3.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_070000_rank4.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_070000_rank5.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_070000_rank6.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_070000_rank7.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_070000_rank0.pt: 83%|████████▎ | 2.07GB / 2.49GB  Processing Files (0 / 9) : 79%|███████▉ | 17.6GB / 22.2GB, 1.06GB/s New Data Upload : 78%|███████▊ | 4.22GB / 5.41GB, 226MB/s  ...eckpoints/model_070000.pt: 36%|███▌ | 834MB / 2.35GB  ...nts/optim_070000_rank1.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_070000_rank2.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_070000_rank3.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_070000_rank4.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_070000_rank5.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_070000_rank6.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_070000_rank7.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_070000_rank0.pt: 84%|████████▍ | 2.08GB / 2.49GB  Processing Files (0 / 9) : 80%|███████▉ | 17.7GB / 22.2GB, 1.06GB/s New Data Upload : 79%|███████▊ | 4.25GB / 5.41GB, 225MB/s  ...eckpoints/model_070000.pt: 37%|███▋ | 862MB / 2.35GB  ...nts/optim_070000_rank1.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_070000_rank2.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_070000_rank3.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_070000_rank4.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_070000_rank5.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_070000_rank6.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_070000_rank7.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_070000_rank0.pt: 84%|████████▍ | 2.09GB / 2.49GB  Processing Files (0 / 9) : 80%|████████ | 17.8GB / 22.2GB, 1.05GB/s New Data Upload : 78%|███████▊ | 4.30GB / 5.48GB, 225MB/s  ...eckpoints/model_070000.pt: 38%|███▊ | 885MB / 2.35GB  ...nts/optim_070000_rank1.pt: 86%|████████▋ | 2.14GB / 2.49GB  ...nts/optim_070000_rank2.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_070000_rank3.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_070000_rank4.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_070000_rank5.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_070000_rank6.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_070000_rank7.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_070000_rank0.pt: 85%|████████▍ | 2.11GB / 2.49GB  Processing Files (0 / 9) : 81%|████████ | 18.0GB / 22.2GB, 1.05GB/s New Data Upload : 79%|███████▉ | 4.34GB / 5.48GB, 227MB/s  ...eckpoints/model_070000.pt: 38%|███▊ | 903MB / 2.35GB  ...nts/optim_070000_rank1.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_070000_rank2.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_070000_rank3.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_070000_rank4.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_070000_rank5.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_070000_rank6.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_070000_rank7.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_070000_rank0.pt: 85%|████████▌ | 2.12GB / 2.49GB  Processing Files (0 / 9) : 81%|████████▏ | 18.1GB / 22.2GB, 1.05GB/s New Data Upload : 79%|███████▉ | 4.37GB / 5.55GB, 227MB/s  ...eckpoints/model_070000.pt: 40%|███▉ | 929MB / 2.35GB  ...nts/optim_070000_rank1.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_070000_rank2.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_070000_rank3.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_070000_rank4.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_070000_rank5.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_070000_rank6.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_070000_rank7.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_070000_rank0.pt: 86%|████████▌ | 2.13GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.2GB / 22.2GB, 1.04GB/s New Data Upload : 80%|███████▉ | 4.42GB / 5.55GB, 226MB/s  ...eckpoints/model_070000.pt: 40%|████ | 950MB / 2.35GB  ...nts/optim_070000_rank1.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_070000_rank2.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_070000_rank3.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_070000_rank4.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_070000_rank5.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_070000_rank6.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_070000_rank7.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_070000_rank0.pt: 86%|████████▌ | 2.13GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.3GB / 22.2GB, 1.03GB/s New Data Upload : 79%|███████▉ | 4.45GB / 5.61GB, 226MB/s  ...eckpoints/model_070000.pt: 42%|████▏ | 983MB / 2.35GB  ...nts/optim_070000_rank1.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_070000_rank2.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_070000_rank3.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_070000_rank4.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_070000_rank5.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_070000_rank6.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_070000_rank7.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_070000_rank0.pt: 86%|████████▋ | 2.15GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.4GB / 22.2GB, 1.02GB/s New Data Upload : 79%|███████▉ | 4.50GB / 5.68GB, 227MB/s  ...eckpoints/model_070000.pt: 43%|████▎ | 1.01GB / 2.35GB  ...nts/optim_070000_rank1.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_070000_rank2.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_070000_rank3.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_070000_rank4.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_070000_rank5.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_070000_rank6.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_070000_rank7.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_070000_rank0.pt: 87%|████████▋ | 2.16GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.5GB / 22.2GB, 1.02GB/s New Data Upload : 80%|███████▉ | 4.54GB / 5.68GB, 227MB/s  ...eckpoints/model_070000.pt: 44%|████▍ | 1.04GB / 2.35GB  ...nts/optim_070000_rank1.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_070000_rank2.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_070000_rank3.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_070000_rank4.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_070000_rank5.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_070000_rank6.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_070000_rank7.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_070000_rank0.pt: 87%|████████▋ | 2.16GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.6GB / 22.2GB, 1.01GB/s New Data Upload : 80%|███████▉ | 4.58GB / 5.75GB, 227MB/s  ...eckpoints/model_070000.pt: 46%|████▌ | 1.07GB / 2.35GB  ...nts/optim_070000_rank1.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_070000_rank2.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_070000_rank3.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_070000_rank4.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_070000_rank5.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_070000_rank6.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_070000_rank7.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_070000_rank0.pt: 87%|████████▋ | 2.17GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▍ | 18.7GB / 22.2GB, 1.00GB/s New Data Upload : 81%|████████ | 4.63GB / 5.75GB, 228MB/s  ...eckpoints/model_070000.pt: 46%|████▋ | 1.09GB / 2.35GB  ...nts/optim_070000_rank1.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_070000_rank2.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_070000_rank3.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_070000_rank4.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_070000_rank5.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_070000_rank6.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_070000_rank7.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_070000_rank0.pt: 88%|████████▊ | 2.18GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▍ | 18.8GB / 22.2GB, 993MB/s New Data Upload : 81%|████████ | 4.67GB / 5.75GB, 227MB/s  ...eckpoints/model_070000.pt: 48%|████▊ | 1.12GB / 2.35GB  ...nts/optim_070000_rank1.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_070000_rank2.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_070000_rank3.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_070000_rank4.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_070000_rank5.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_070000_rank6.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_070000_rank7.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_070000_rank0.pt: 88%|████████▊ | 2.19GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▍ | 18.9GB / 22.2GB, 985MB/s New Data Upload : 81%|████████ | 4.71GB / 5.81GB, 227MB/s  ...eckpoints/model_070000.pt: 49%|████▉ | 1.15GB / 2.35GB  ...nts/optim_070000_rank1.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_070000_rank2.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_070000_rank3.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_070000_rank4.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_070000_rank5.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_070000_rank6.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_070000_rank7.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_070000_rank0.pt: 89%|████████▊ | 2.20GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▌ | 19.0GB / 22.2GB, 980MB/s New Data Upload : 82%|████████▏ | 4.75GB / 5.81GB, 228MB/s  ...eckpoints/model_070000.pt: 50%|████▉ | 1.17GB / 2.35GB  ...nts/optim_070000_rank1.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_070000_rank2.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_070000_rank3.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_070000_rank4.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_070000_rank5.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_070000_rank6.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_070000_rank7.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_070000_rank0.pt: 89%|████████▉ | 2.21GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▌ | 19.1GB / 22.2GB, 968MB/s New Data Upload : 81%|████████▏ | 4.79GB / 5.88GB, 228MB/s  ...eckpoints/model_070000.pt: 51%|█████ | 1.19GB / 2.35GB  ...nts/optim_070000_rank1.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_070000_rank2.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_070000_rank3.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_070000_rank4.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_070000_rank5.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_070000_rank6.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_070000_rank7.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_070000_rank0.pt: 89%|████████▉ | 2.22GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▋ | 19.2GB / 22.2GB, 965MB/s New Data Upload : 82%|████████▏ | 4.83GB / 5.88GB, 228MB/s  ...eckpoints/model_070000.pt: 52%|█████▏ | 1.23GB / 2.35GB  ...nts/optim_070000_rank1.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_070000_rank2.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_070000_rank3.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_070000_rank4.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_070000_rank5.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_070000_rank6.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_070000_rank7.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_070000_rank0.pt: 90%|████████▉ | 2.23GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.3GB / 22.2GB, 956MB/s New Data Upload : 82%|████████▏ | 4.88GB / 5.95GB, 228MB/s  ...eckpoints/model_070000.pt: 53%|█████▎ | 1.25GB / 2.35GB  ...nts/optim_070000_rank1.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_070000_rank2.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_070000_rank3.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_070000_rank4.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_070000_rank5.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_070000_rank6.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_070000_rank7.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_070000_rank0.pt: 90%|█████████ | 2.24GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.4GB / 22.2GB, 948MB/s New Data Upload : 82%|████████▏ | 4.92GB / 6.01GB, 227MB/s  ...eckpoints/model_070000.pt: 54%|█████▍ | 1.28GB / 2.35GB  ...nts/optim_070000_rank1.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_070000_rank2.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_070000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_070000_rank4.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_070000_rank5.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_070000_rank6.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_070000_rank7.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_070000_rank0.pt: 91%|█████████ | 2.25GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.5GB / 22.2GB, 937MB/s New Data Upload : 83%|████████▎ | 4.96GB / 6.01GB, 227MB/s  ...eckpoints/model_070000.pt: 56%|█████▌ | 1.31GB / 2.35GB  ...nts/optim_070000_rank1.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_070000_rank2.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_070000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_070000_rank4.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_070000_rank5.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_070000_rank6.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_070000_rank7.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_070000_rank0.pt: 91%|█████████ | 2.26GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.6GB / 22.2GB, 925MB/s New Data Upload : 83%|████████▎ | 5.01GB / 6.01GB, 226MB/s  ...eckpoints/model_070000.pt: 57%|█████▋ | 1.34GB / 2.35GB  ...nts/optim_070000_rank1.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_070000_rank2.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_070000_rank3.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_070000_rank4.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_070000_rank5.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_070000_rank6.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_070000_rank7.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_070000_rank0.pt: 91%|█████████▏| 2.27GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▊ | 19.7GB / 22.2GB, 915MB/s New Data Upload : 83%|████████▎ | 5.05GB / 6.08GB, 226MB/s  ...eckpoints/model_070000.pt: 59%|█████▉ | 1.38GB / 2.35GB  ...nts/optim_070000_rank1.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_070000_rank2.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_070000_rank3.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_070000_rank4.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_070000_rank5.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_070000_rank6.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_070000_rank7.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_070000_rank0.pt: 92%|█████████▏| 2.28GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▉ | 19.8GB / 22.2GB, 908MB/s New Data Upload : 83%|████████▎ | 5.10GB / 6.15GB, 227MB/s  ...eckpoints/model_070000.pt: 61%|██████ | 1.42GB / 2.35GB  ...nts/optim_070000_rank1.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_070000_rank2.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_070000_rank3.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_070000_rank4.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_070000_rank5.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_070000_rank6.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_070000_rank7.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_070000_rank0.pt: 92%|█████████▏| 2.29GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▉ | 19.9GB / 22.2GB, 895MB/s New Data Upload : 84%|████████▍ | 5.16GB / 6.15GB, 227MB/s  ...eckpoints/model_070000.pt: 62%|██████▏ | 1.46GB / 2.35GB  ...nts/optim_070000_rank1.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_070000_rank2.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_070000_rank3.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_070000_rank4.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_070000_rank5.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_070000_rank6.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_070000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_070000_rank0.pt: 92%|█████████▏| 2.30GB / 2.49GB  Processing Files (0 / 9) : 90%|████████▉ | 20.0GB / 22.2GB, 886MB/s New Data Upload : 84%|████████▍ | 5.21GB / 6.22GB, 227MB/s  ...eckpoints/model_070000.pt: 64%|██████▎ | 1.49GB / 2.35GB  ...nts/optim_070000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_070000_rank2.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_070000_rank3.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_070000_rank4.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_070000_rank5.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_070000_rank6.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_070000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_070000_rank0.pt: 93%|█████████▎| 2.31GB / 2.49GB  Processing Files (0 / 9) : 90%|█████████ | 20.1GB / 22.2GB, 877MB/s New Data Upload : 85%|████████▍ | 5.25GB / 6.22GB, 227MB/s  ...eckpoints/model_070000.pt: 65%|██████▌ | 1.53GB / 2.35GB  ...nts/optim_070000_rank1.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_070000_rank2.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_070000_rank3.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_070000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_070000_rank5.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_070000_rank6.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_070000_rank7.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_070000_rank0.pt: 93%|█████████▎| 2.32GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.2GB / 22.2GB, 871MB/s New Data Upload : 84%|████████▍ | 5.30GB / 6.28GB, 229MB/s  ...eckpoints/model_070000.pt: 67%|██████▋ | 1.57GB / 2.35GB  ...nts/optim_070000_rank1.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_070000_rank2.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_070000_rank3.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_070000_rank4.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_070000_rank5.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_070000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_070000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_070000_rank0.pt: 94%|█████████▎| 2.32GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.3GB / 22.2GB, 860MB/s New Data Upload : 85%|████████▌ | 5.35GB / 6.28GB, 228MB/s  ...eckpoints/model_070000.pt: 68%|██████▊ | 1.60GB / 2.35GB  ...nts/optim_070000_rank1.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_070000_rank2.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_070000_rank3.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_070000_rank4.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_070000_rank5.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_070000_rank6.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_070000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_070000_rank0.pt: 94%|█████████▎| 2.33GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.4GB / 22.2GB, 851MB/s New Data Upload : 85%|████████▍ | 5.39GB / 6.35GB, 229MB/s  ...eckpoints/model_070000.pt: 70%|██████▉ | 1.64GB / 2.35GB  ...nts/optim_070000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_070000_rank2.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_070000_rank3.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_070000_rank4.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_070000_rank5.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_070000_rank6.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_070000_rank7.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_070000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.5GB / 22.2GB, 845MB/s New Data Upload : 86%|████████▌ | 5.45GB / 6.36GB, 230MB/s  ...eckpoints/model_070000.pt: 71%|███████▏ | 1.68GB / 2.35GB  ...nts/optim_070000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_070000_rank2.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_070000_rank3.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_070000_rank4.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_070000_rank5.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_070000_rank6.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_070000_rank7.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_070000_rank0.pt: 94%|█████████▍| 2.35GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.5GB / 22.2GB, 826MB/s New Data Upload : 86%|████████▋ | 5.49GB / 6.36GB, 230MB/s  ...eckpoints/model_070000.pt: 72%|███████▏ | 1.70GB / 2.35GB  ...nts/optim_070000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_070000_rank2.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_070000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_070000_rank4.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_070000_rank5.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_070000_rank6.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_070000_rank7.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_070000_rank0.pt: 95%|█████████▍| 2.36GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.7GB / 22.2GB, 803MB/s New Data Upload : 87%|████████▋ | 5.53GB / 6.36GB, 230MB/s  ...eckpoints/model_070000.pt: 74%|███████▎ | 1.73GB / 2.35GB  ...nts/optim_070000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_070000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_070000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_070000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_070000_rank5.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_070000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_070000_rank7.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_070000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.7GB / 22.2GB, 776MB/s New Data Upload : 88%|████████▊ | 5.57GB / 6.36GB, 229MB/s  ...eckpoints/model_070000.pt: 74%|███████▍ | 1.75GB / 2.35GB  ...nts/optim_070000_rank1.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_070000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_070000_rank3.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_070000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_070000_rank5.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_070000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_070000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_070000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.8GB / 22.2GB, 755MB/s New Data Upload : 88%|████████▊ | 5.60GB / 6.36GB, 227MB/s  ...eckpoints/model_070000.pt: 75%|███████▌ | 1.77GB / 2.35GB  ...nts/optim_070000_rank1.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_070000_rank2.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_070000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_070000_rank4.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_070000_rank5.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_070000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_070000_rank7.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_070000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 20.8GB / 22.2GB, 734MB/s New Data Upload : 89%|████████▊ | 5.63GB / 6.36GB, 225MB/s  ...eckpoints/model_070000.pt: 76%|███████▌ | 1.79GB / 2.35GB  ...nts/optim_070000_rank1.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank2.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_070000_rank3.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_070000_rank4.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_070000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_070000_rank7.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_070000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 20.9GB / 22.2GB, 698MB/s New Data Upload : 89%|████████▉ | 5.66GB / 6.36GB, 223MB/s  ...eckpoints/model_070000.pt: 77%|███████▋ | 1.81GB / 2.35GB  ...nts/optim_070000_rank1.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank2.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank3.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_070000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_070000_rank6.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank7.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank0.pt: 96%|█████████▌| 2.39GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 21.0GB / 22.2GB, 677MB/s New Data Upload : 89%|████████▉ | 5.69GB / 6.36GB, 222MB/s  ...eckpoints/model_070000.pt: 78%|███████▊ | 1.83GB / 2.35GB  ...nts/optim_070000_rank1.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank2.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank3.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank5.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_070000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank0.pt: 96%|█████████▋| 2.39GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▍| 21.0GB / 22.2GB, 653MB/s New Data Upload : 90%|████████▉ | 5.72GB / 6.36GB, 220MB/s  ...eckpoints/model_070000.pt: 79%|███████▊ | 1.85GB / 2.35GB  ...nts/optim_070000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank2.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank3.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank5.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_070000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank0.pt: 97%|█████████▋| 2.40GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▍| 21.1GB / 22.2GB, 631MB/s New Data Upload : 90%|█████████ | 5.74GB / 6.36GB, 218MB/s  ...eckpoints/model_070000.pt: 79%|███████▉ | 1.86GB / 2.35GB  ...nts/optim_070000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank3.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank5.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank0.pt: 97%|█████████▋| 2.40GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▍| 21.1GB / 22.2GB, 608MB/s New Data Upload : 91%|█████████ | 5.77GB / 6.36GB, 215MB/s  ...eckpoints/model_070000.pt: 80%|████████ | 1.88GB / 2.35GB  ...nts/optim_070000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank5.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.1GB / 22.2GB, 584MB/s New Data Upload : 91%|█████████ | 5.79GB / 6.36GB, 212MB/s  ...eckpoints/model_070000.pt: 81%|████████ | 1.90GB / 2.35GB  ...nts/optim_070000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_070000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_070000_rank5.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_070000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_070000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_070000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.2GB / 22.2GB, 565MB/s New Data Upload : 91%|█████████▏| 5.82GB / 6.36GB, 211MB/s  ...eckpoints/model_070000.pt: 82%|████████▏ | 1.91GB / 2.35GB  ...nts/optim_070000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_070000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_070000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_070000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_070000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_070000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 541MB/s New Data Upload : 92%|█████████▏| 5.84GB / 6.36GB, 206MB/s  ...eckpoints/model_070000.pt: 82%|████████▏ | 1.93GB / 2.35GB  ...nts/optim_070000_rank1.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_070000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_070000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_070000_rank4.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_070000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_070000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_070000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_070000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 522MB/s New Data Upload : 92%|█████████▏| 5.85GB / 6.36GB, 203MB/s  ...eckpoints/model_070000.pt: 83%|████████▎ | 1.94GB / 2.35GB  ...nts/optim_070000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank2.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_070000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_070000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_070000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 509MB/s New Data Upload : 92%|█████████▏| 5.87GB / 6.36GB, 201MB/s  ...eckpoints/model_070000.pt: 83%|████████▎ | 1.96GB / 2.35GB  ...nts/optim_070000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_070000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.4GB / 22.2GB, 491MB/s New Data Upload : 93%|█████████▎| 5.89GB / 6.36GB, 198MB/s  ...eckpoints/model_070000.pt: 84%|████████▍ | 1.97GB / 2.35GB  ...nts/optim_070000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▋| 21.4GB / 22.2GB, 469MB/s New Data Upload : 93%|█████████▎| 5.91GB / 6.36GB, 194MB/s  ...eckpoints/model_070000.pt: 84%|████████▍ | 1.98GB / 2.35GB  ...nts/optim_070000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_070000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_070000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_070000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_070000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_070000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_070000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_070000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 456MB/s New Data Upload : 93%|█████████▎| 5.93GB / 6.36GB, 191MB/s  ...eckpoints/model_070000.pt: 85%|████████▍ | 1.99GB / 2.35GB  ...nts/optim_070000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_070000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_070000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_070000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_070000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_070000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_070000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_070000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 437MB/s New Data Upload : 94%|█████████▎| 5.95GB / 6.36GB, 188MB/s  ...eckpoints/model_070000.pt: 86%|████████▌ | 2.01GB / 2.35GB  ...nts/optim_070000_rank1.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_070000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_070000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_070000_rank6.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank7.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 423MB/s New Data Upload : 94%|█████████▍| 5.97GB / 6.36GB, 185MB/s  ...eckpoints/model_070000.pt: 86%|████████▋ | 2.03GB / 2.35GB  ...nts/optim_070000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 414MB/s New Data Upload : 94%|█████████▍| 5.99GB / 6.36GB, 182MB/s  ...eckpoints/model_070000.pt: 87%|████████▋ | 2.04GB / 2.35GB  ...nts/optim_070000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_070000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.7GB / 22.2GB, 407MB/s New Data Upload : 95%|█████████▍| 6.01GB / 6.36GB, 180MB/s  ...eckpoints/model_070000.pt: 87%|████████▋ | 2.05GB / 2.35GB  ...nts/optim_070000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank5.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...nts/optim_070000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 400MB/s New Data Upload : 95%|█████████▍| 6.02GB / 6.36GB, 177MB/s  ...eckpoints/model_070000.pt: 88%|████████▊ | 2.07GB / 2.35GB  ...nts/optim_070000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 392MB/s New Data Upload : 95%|█████████▌| 6.05GB / 6.36GB, 176MB/s  ...eckpoints/model_070000.pt: 88%|████████▊ | 2.07GB / 2.35GB  ...nts/optim_070000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 386MB/s New Data Upload : 95%|█████████▌| 6.06GB / 6.36GB, 173MB/s  ...eckpoints/model_070000.pt: 89%|████████▉ | 2.09GB / 2.35GB  ...nts/optim_070000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 372MB/s New Data Upload : 95%|█████████▌| 6.07GB / 6.36GB, 169MB/s  ...eckpoints/model_070000.pt: 89%|████████▉ | 2.10GB / 2.35GB  ...nts/optim_070000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 367MB/s New Data Upload : 96%|█████████▌| 6.09GB / 6.36GB, 168MB/s  ...eckpoints/model_070000.pt: 90%|████████▉ | 2.11GB / 2.35GB  ...nts/optim_070000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_070000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 358MB/s New Data Upload : 96%|█████████▌| 6.10GB / 6.36GB, 165MB/s  ...eckpoints/model_070000.pt: 90%|█████████ | 2.12GB / 2.35GB  ...nts/optim_070000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 355MB/s New Data Upload : 96%|█████████▌| 6.11GB / 6.36GB, 163MB/s  ...eckpoints/model_070000.pt: 91%|█████████ | 2.13GB / 2.35GB  ...nts/optim_070000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 346MB/s New Data Upload : 96%|█████████▋| 6.12GB / 6.36GB, 159MB/s  ...eckpoints/model_070000.pt: 91%|█████████ | 2.14GB / 2.35GB  ...nts/optim_070000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 336MB/s New Data Upload : 96%|█████████▋| 6.13GB / 6.36GB, 156MB/s  ...eckpoints/model_070000.pt: 91%|█████████▏| 2.15GB / 2.35GB  ...nts/optim_070000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 330MB/s New Data Upload : 97%|█████████▋| 6.14GB / 6.36GB, 153MB/s  ...eckpoints/model_070000.pt: 92%|█████████▏| 2.16GB / 2.35GB  ...nts/optim_070000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 321MB/s New Data Upload : 97%|█████████▋| 6.15GB / 6.36GB, 149MB/s  ...eckpoints/model_070000.pt: 92%|█████████▏| 2.17GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 313MB/s New Data Upload : 97%|█████████▋| 6.16GB / 6.36GB, 147MB/s  ...eckpoints/model_070000.pt: 93%|█████████▎| 2.18GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 304MB/s New Data Upload : 97%|█████████▋| 6.17GB / 6.36GB, 144MB/s  ...eckpoints/model_070000.pt: 93%|█████████▎| 2.18GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 296MB/s New Data Upload : 97%|█████████▋| 6.18GB / 6.36GB, 140MB/s  ...eckpoints/model_070000.pt: 93%|█████████▎| 2.19GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 288MB/s New Data Upload : 97%|█████████▋| 6.19GB / 6.36GB, 137MB/s  ...eckpoints/model_070000.pt: 94%|█████████▎| 2.20GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 279MB/s New Data Upload : 97%|█████████▋| 6.20GB / 6.36GB, 134MB/s  ...eckpoints/model_070000.pt: 94%|█████████▍| 2.21GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 271MB/s New Data Upload : 98%|█████████▊| 6.21GB / 6.36GB, 130MB/s  ...eckpoints/model_070000.pt: 94%|█████████▍| 2.21GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 263MB/s New Data Upload : 98%|█████████▊| 6.22GB / 6.36GB, 128MB/s  ...eckpoints/model_070000.pt: 95%|█████████▍| 2.22GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 253MB/s New Data Upload : 98%|█████████▊| 6.23GB / 6.36GB, 124MB/s  ...eckpoints/model_070000.pt: 95%|█████████▍| 2.23GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 247MB/s New Data Upload : 98%|█████████▊| 6.24GB / 6.36GB, 121MB/s  ...eckpoints/model_070000.pt: 95%|█████████▌| 2.24GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 237MB/s New Data Upload : 98%|█████████▊| 6.25GB / 6.36GB, 117MB/s  ...eckpoints/model_070000.pt: 96%|█████████▌| 2.25GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 228MB/s New Data Upload : 98%|█████████▊| 6.25GB / 6.36GB, 113MB/s  ...eckpoints/model_070000.pt: 96%|█████████▌| 2.25GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 220MB/s New Data Upload : 99%|█████████▊| 6.26GB / 6.36GB, 108MB/s  ...eckpoints/model_070000.pt: 96%|█████████▋| 2.26GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 211MB/s New Data Upload : 99%|█████████▊| 6.27GB / 6.36GB, 104MB/s  ...eckpoints/model_070000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 203MB/s New Data Upload : 99%|█████████▉| 6.28GB / 6.36GB, 101MB/s  ...eckpoints/model_070000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 193MB/s New Data Upload : 99%|█████████▉| 6.28GB / 6.36GB, 96.1MB/s  ...eckpoints/model_070000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 184MB/s New Data Upload : 99%|█████████▉| 6.29GB / 6.36GB, 91.9MB/s  ...eckpoints/model_070000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 178MB/s New Data Upload : 99%|█████████▉| 6.29GB / 6.36GB, 88.2MB/s  ...eckpoints/model_070000.pt: 97%|█████████▋| 2.29GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 167MB/s New Data Upload : 99%|█████████▉| 6.30GB / 6.36GB, 83.1MB/s  ...eckpoints/model_070000.pt: 98%|█████████▊| 2.29GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 160MB/s New Data Upload : 99%|█████████▉| 6.30GB / 6.36GB, 79.2MB/s  ...eckpoints/model_070000.pt: 98%|█████████▊| 2.29GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 150MB/s New Data Upload : 99%|█████████▉| 6.30GB / 6.36GB, 75.5MB/s  ...eckpoints/model_070000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 144MB/s New Data Upload : 99%|█████████▉| 6.31GB / 6.36GB, 72.4MB/s  ...eckpoints/model_070000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 140MB/s New Data Upload : 99%|█████████▉| 6.31GB / 6.36GB, 70.4MB/s  ...eckpoints/model_070000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 132MB/s New Data Upload : 99%|█████████▉| 6.32GB / 6.36GB, 67.4MB/s  ...eckpoints/model_070000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 126MB/s New Data Upload : 99%|█████████▉| 6.32GB / 6.36GB, 64.8MB/s  ...eckpoints/model_070000.pt: 99%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 121MB/s New Data Upload : 99%|█████████▉| 6.32GB / 6.36GB, 62.4MB/s  ...eckpoints/model_070000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 116MB/s New Data Upload : 100%|█████████▉| 6.33GB / 6.36GB, 60.0MB/s  ...eckpoints/model_070000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 112MB/s New Data Upload : 100%|█████████▉| 6.33GB / 6.36GB, 57.8MB/s  ...eckpoints/model_070000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 108MB/s New Data Upload : 100%|█████████▉| 6.33GB / 6.36GB, 55.7MB/s  ...eckpoints/model_070000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 104MB/s New Data Upload : 100%|█████████▉| 6.34GB / 6.36GB, 53.8MB/s  ...eckpoints/model_070000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 99.2MB/s New Data Upload : 100%|█████████▉| 6.34GB / 6.36GB, 51.3MB/s  ...eckpoints/model_070000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 94.5MB/s New Data Upload : 100%|█████████▉| 6.34GB / 6.36GB, 49.5MB/s  ...eckpoints/model_070000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 91.0MB/s New Data Upload : 100%|█████████▉| 6.34GB / 6.36GB, 48.3MB/s  ...eckpoints/model_070000.pt: 99%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 86.8MB/s New Data Upload : 100%|█████████▉| 6.35GB / 6.36GB, 46.5MB/s  ...eckpoints/model_070000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 82.1MB/s New Data Upload : 100%|█████████▉| 6.35GB / 6.36GB, 44.7MB/s  ...eckpoints/model_070000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 78.3MB/s New Data Upload : 100%|█████████▉| 6.35GB / 6.36GB, 43.3MB/s  ...eckpoints/model_070000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 73.4MB/s New Data Upload : 100%|█████████▉| 6.35GB / 6.36GB, 41.4MB/s  ...eckpoints/model_070000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 69.3MB/s New Data Upload : 100%|█████████▉| 6.36GB / 6.36GB, 40.0MB/s  ...eckpoints/model_070000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 65.4MB/s New Data Upload : 100%|█████████▉| 6.36GB / 6.36GB, 38.1MB/s  ...eckpoints/model_070000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 59.5MB/s New Data Upload : 100%|█████████▉| 6.36GB / 6.36GB, 35.9MB/s  ...eckpoints/model_070000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 51.8MB/s New Data Upload : 100%|█████████▉| 6.36GB / 6.36GB, 32.7MB/s  ...eckpoints/model_070000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (9 / 9) : 100%|██████████| 22.2GB / 22.2GB, 35.7MB/s New Data Upload : 100%|██████████| 6.36GB / 6.36GB, 24.0MB/s  ...eckpoints/model_070000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_070000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (9 / 9) : 100%|██████████| 22.2GB / 22.2GB, 13.8MB/s New Data Upload : 100%|██████████| 6.36GB / 6.36GB, 12.2MB/s ...eckpoints/model_070000.pt: 100%|██████████| 2.35GB / 2.35GB ...nts/optim_070000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_070000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_070000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_070000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_070000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_070000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_070000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_070000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB [19:58:30] Step 70000 uploaded [20:58:30] Uploading step 75000... Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...nts/optim_075000_rank0.pt: 0%| | 27.6kB / 2.49GB  ...nts/optim_075000_rank1.pt: 0%| | 584kB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 97.7kB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 96.6kB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 27.6kB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 27.6kB / 2.49GB  ...nts/optim_075000_rank0.pt: 0%| | 27.6kB / 2.49GB  ...nts/optim_075000_rank1.pt: 0%| | 584kB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 97.7kB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 96.6kB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 27.6kB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 27.6kB / 2.49GB  Processing Files (0 / 6) : 0%| | 861kB / 22.2GB, 1.08MB/s New Data Upload : 0%| | 584kB / 604MB, 730kB/s  ...nts/optim_075000_rank0.pt: 0%| | 27.6kB / 2.49GB  ...nts/optim_075000_rank1.pt: 0%| | 584kB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 97.7kB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 96.6kB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 27.6kB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 27.6kB / 2.49GB  ...nts/optim_075000_rank0.pt: 0%| | 110kB / 2.49GB  ...nts/optim_075000_rank1.pt: 0%| | 2.34MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 391kB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 386kB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 110kB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 110kB / 2.49GB  Processing Files (0 / 6) : 0%| | 3.44MB / 22.2GB, 2.87MB/s New Data Upload : 0%| | 2.34MB / 604MB, 1.95MB/s  ...nts/optim_075000_rank0.pt: 0%| | 249kB / 2.49GB  ...nts/optim_075000_rank1.pt: 0%| | 5.26MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 879kB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 869kB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 249kB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 249kB / 2.49GB  Processing Files (0 / 6) : 0%| | 7.75MB / 22.2GB, 5.53MB/s New Data Upload : 1%| | 5.26MB / 604MB, 3.75MB/s  ...nts/optim_075000_rank0.pt: 0%| | 387kB / 2.49GB  ...nts/optim_075000_rank1.pt: 0%| | 8.18MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 1.37MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 1.35MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 387kB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 387kB / 2.49GB  Processing Files (0 / 6) : 0%| | 12.1MB / 22.2GB, 7.53MB/s New Data Upload : 1%|▏ | 8.18MB / 604MB, 5.11MB/s  ...nts/optim_075000_rank0.pt: 0%| | 497kB / 2.49GB  ...nts/optim_075000_rank1.pt: 0%| | 10.5MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 1.76MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 1.74MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 497kB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 497kB / 2.49GB  Processing Files (0 / 6) : 0%| | 15.5MB / 22.2GB, 8.61MB/s New Data Upload : 2%|▏ | 10.5MB / 604MB, 5.84MB/s  ...nts/optim_075000_rank0.pt: 0%| | 829kB / 2.49GB  ...nts/optim_075000_rank1.pt: 1%| | 17.5MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 2.93MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 2.90MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 829kB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 829kB / 2.49GB  Processing Files (0 / 6) : 0%| | 25.8MB / 22.2GB, 12.9MB/s New Data Upload : 3%|▎ | 17.5MB / 604MB, 8.76MB/s  ...nts/optim_075000_rank0.pt: 0%| | 1.02MB / 2.49GB  ...nts/optim_075000_rank1.pt: 1%| | 21.6MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 3.61MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 3.57MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 1.02MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 1.02MB / 2.49GB  Processing Files (0 / 6) : 0%| | 31.9MB / 22.2GB, 14.5MB/s New Data Upload : 4%|▎ | 21.6MB / 604MB, 9.82MB/s  ...nts/optim_075000_rank0.pt: 0%| | 1.10MB / 2.49GB  ...nts/optim_075000_rank1.pt: 1%| | 23.4MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 3.91MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 3.86MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 1.10MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 1.10MB / 2.49GB  Processing Files (0 / 6) : 0%| | 34.4MB / 22.2GB, 14.3MB/s New Data Upload : 4%|▍ | 23.4MB / 604MB, 9.73MB/s  ...nts/optim_075000_rank0.pt: 0%| | 1.41MB / 2.49GB  ...nts/optim_075000_rank1.pt: 1%| | 29.8MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 4.98MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 4.93MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 1.41MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 1.41MB / 2.49GB  Processing Files (0 / 6) : 0%| | 43.9MB / 22.2GB, 16.9MB/s New Data Upload : 5%|▍ | 29.8MB / 604MB, 11.5MB/s  ...nts/optim_075000_rank0.pt: 0%| | 1.63MB / 2.49GB  ...nts/optim_075000_rank1.pt: 1%|▏ | 34.5MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 5.76MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 5.70MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 1.63MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 1.63MB / 2.49GB  Processing Files (0 / 6) : 0%| | 50.8MB / 22.2GB, 18.1MB/s New Data Upload : 6%|▌ | 34.5MB / 604MB, 12.3MB/s  ...nts/optim_075000_rank0.pt: 0%| | 1.82MB / 2.49GB  ...nts/optim_075000_rank1.pt: 2%|▏ | 38.5MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 6.45MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 6.37MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 1.82MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 1.82MB / 2.49GB  Processing Files (0 / 6) : 0%| | 56.8MB / 22.2GB, 18.9MB/s New Data Upload : 6%|▋ | 38.5MB / 604MB, 12.8MB/s  ...nts/optim_075000_rank0.pt: 0%| | 1.93MB / 2.49GB  ...nts/optim_075000_rank1.pt: 2%|▏ | 40.9MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 6.84MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 6.76MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 1.93MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 1.93MB / 2.49GB  Processing Files (0 / 6) : 0%| | 60.3MB / 22.2GB, 18.8MB/s New Data Upload : 7%|▋ | 40.9MB / 604MB, 12.8MB/s  ...nts/optim_075000_rank0.pt: 0%| | 2.15MB / 2.49GB  ...nts/optim_075000_rank1.pt: 2%|▏ | 45.5MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 7.62MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 7.53MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 2.15MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 2.15MB / 2.49GB  Processing Files (0 / 6) : 0%| | 67.2MB / 22.2GB, 19.8MB/s New Data Upload : 8%|▊ | 45.5MB / 604MB, 13.4MB/s  ...nts/optim_075000_rank0.pt: 0%| | 2.38MB / 2.49GB  ...nts/optim_075000_rank1.pt: 2%|▏ | 50.2MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 8.40MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 8.31MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 2.38MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 2.38MB / 2.49GB  Processing Files (0 / 6) : 0%| | 74.1MB / 22.2GB, 20.6MB/s New Data Upload : 8%|▊ | 50.2MB / 604MB, 13.9MB/s  ...nts/optim_075000_rank0.pt: 0%| | 2.60MB / 2.49GB  ...nts/optim_075000_rank1.pt: 2%|▏ | 54.9MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 9.18MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 9.08MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 2.60MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 2.60MB / 2.49GB  Processing Files (0 / 6) : 0%| | 80.9MB / 22.2GB, 21.3MB/s New Data Upload : 9%|▉ | 54.9MB / 604MB, 14.4MB/s  ...nts/optim_075000_rank0.pt: 0%| | 2.85MB / 2.49GB  ...nts/optim_075000_rank1.pt: 2%|▏ | 60.1MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 10.1MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 9.95MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 2.85MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 2.85MB / 2.49GB  Processing Files (0 / 6) : 0%| | 88.7MB / 22.2GB, 22.2MB/s New Data Upload : 10%|▉ | 60.1MB / 604MB, 15.0MB/s  ...nts/optim_075000_rank0.pt: 0%| | 2.96MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 62.5MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 10.5MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 10.3MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 2.96MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 2.96MB / 2.49GB  Processing Files (0 / 6) : 0%| | 92.1MB / 22.2GB, 21.9MB/s New Data Upload : 10%|█ | 62.5MB / 604MB, 14.9MB/s  ...nts/optim_075000_rank0.pt: 0%| | 3.15MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 66.6MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 11.0MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 3.15MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 3.15MB / 2.49GB  Processing Files (0 / 6) : 0%| | 98.2MB / 22.2GB, 22.3MB/s New Data Upload : 11%|█ | 66.6MB / 604MB, 15.1MB/s  ...nts/optim_075000_rank0.pt: 0%| | 3.15MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 66.6MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 11.0MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 3.15MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 3.15MB / 2.49GB  ...nts/optim_075000_rank0.pt: 0%| | 3.15MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 66.6MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 11.0MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 3.15MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 3.15MB / 2.49GB  ...nts/optim_075000_rank0.pt: 0%| | 3.15MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 66.6MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 11.0MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 3.15MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 3.15MB / 2.49GB  ...nts/optim_075000_rank0.pt: 0%| | 3.15MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 66.6MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 11.0MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 3.15MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 3.15MB / 2.49GB  ...nts/optim_075000_rank6.pt: 0%| | 3.25MB / 2.49GB  ...nts/optim_075000_rank0.pt: 0%| | 3.85MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 67.2MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 11.3MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 11.7MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 7.26MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 3.74MB / 2.49GB  ...nts/optim_075000_rank6.pt: 0%| | 3.25MB / 2.49GB  Processing Files (0 / 7) : 0%| | 108MB / 22.2GB, 20.0MB/s New Data Upload : 12%|█▏ | 71.2MB / 604MB, 13.2MB/s  ...nts/optim_075000_rank0.pt: 0%| | 4.30MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 67.2MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 11.4MB / 2.49GB  ...nts/optim_075000_rank4.pt: 0%| | 12.1MB / 2.49GB  ...nts/optim_075000_rank5.pt: 0%| | 9.11MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 4.22MB / 2.49GB  ...nts/optim_075000_rank6.pt: 0%| | 3.87MB / 2.49GB  Processing Files (0 / 7) : 1%| | 112MB / 22.2GB, 20.0MB/s New Data Upload : 11%|█ | 73.5MB / 671MB, 13.1MB/s  ...nts/optim_075000_rank0.pt: 0%| | 5.17MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 67.3MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 11.5MB / 2.49GB  ...nts/optim_075000_rank4.pt: 1%| | 12.8MB / 2.49GB  ...nts/optim_075000_rank5.pt: 1%| | 14.4MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 4.95MB / 2.49GB  ...nts/optim_075000_rank6.pt: 0%| | 3.98MB / 2.49GB  Processing Files (0 / 7) : 1%| | 120MB / 22.2GB, 20.7MB/s New Data Upload : 11%|█ | 78.8MB / 738MB, 13.6MB/s  ...nts/optim_075000_rank0.pt: 0%| | 5.97MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 67.4MB / 2.49GB  ...nts/optim_075000_rank3.pt: 0%| | 12.4MB / 2.49GB  ...nts/optim_075000_rank4.pt: 1%| | 13.5MB / 2.49GB  ...nts/optim_075000_rank5.pt: 1%| | 16.9MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 5.74MB / 2.49GB  ...nts/optim_075000_rank6.pt: 0%| | 5.26MB / 2.49GB  Processing Files (0 / 7) : 1%| | 127MB / 22.2GB, 21.2MB/s New Data Upload : 11%|█ | 82.9MB / 738MB, 13.8MB/s  ...nts/optim_075000_rank0.pt: 0%| | 7.74MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 67.5MB / 2.49GB  ...nts/optim_075000_rank3.pt: 1%| | 13.0MB / 2.49GB  ...nts/optim_075000_rank4.pt: 1%| | 15.1MB / 2.49GB  ...nts/optim_075000_rank5.pt: 1%| | 22.1MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 7.81MB / 2.49GB  ...nts/optim_075000_rank6.pt: 0%| | 8.86MB / 2.49GB  Processing Files (0 / 7) : 1%| | 142MB / 22.2GB, 22.9MB/s New Data Upload : 12%|█▏ | 91.0MB / 738MB, 14.7MB/s  ...nts/optim_075000_rank0.pt: 0%| | 9.90MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 67.7MB / 2.49GB  ...nts/optim_075000_rank3.pt: 1%| | 14.6MB / 2.49GB  ...nts/optim_075000_rank4.pt: 1%| | 17.1MB / 2.49GB  ...nts/optim_075000_rank5.pt: 1%| | 26.5MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 10.4MB / 2.49GB  ...nts/optim_075000_rank6.pt: 1%| | 14.2MB / 2.49GB  Processing Files (0 / 7) : 1%| | 160MB / 22.2GB, 25.1MB/s New Data Upload : 14%|█▎ | 100MB / 738MB, 15.7MB/s  ...nts/optim_075000_rank0.pt: 0%| | 11.4MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 68.2MB / 2.49GB  ...nts/optim_075000_rank3.pt: 1%| | 17.3MB / 2.49GB  ...nts/optim_075000_rank4.pt: 1%| | 18.3MB / 2.49GB  ...nts/optim_075000_rank5.pt: 1%| | 30.5MB / 2.49GB  ...nts/optim_075000_rank7.pt: 0%| | 12.2MB / 2.49GB  ...nts/optim_075000_rank6.pt: 1%| | 16.5MB / 2.49GB  Processing Files (0 / 7) : 1%| | 174MB / 22.2GB, 26.4MB/s New Data Upload : 13%|█▎ | 109MB / 805MB, 16.4MB/s  ...nts/optim_075000_rank0.pt: 1%| | 14.0MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 68.8MB / 2.49GB  ...nts/optim_075000_rank3.pt: 1%| | 22.9MB / 2.49GB  ...nts/optim_075000_rank4.pt: 1%| | 20.3MB / 2.49GB  ...nts/optim_075000_rank5.pt: 1%|▏ | 34.8MB / 2.49GB  ...nts/optim_075000_rank7.pt: 1%| | 14.6MB / 2.49GB  ...nts/optim_075000_rank6.pt: 1%| | 21.8MB / 2.49GB  Processing Files (0 / 7) : 1%| | 197MB / 22.2GB, 29.0MB/s New Data Upload : 15%|█▌ | 121MB / 805MB, 17.9MB/s  ...nts/optim_075000_rank0.pt: 1%| | 16.0MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 69.7MB / 2.49GB  ...nts/optim_075000_rank3.pt: 1%| | 28.4MB / 2.49GB  ...nts/optim_075000_rank4.pt: 1%| | 21.9MB / 2.49GB  ...nts/optim_075000_rank5.pt: 1%|▏ | 35.8MB / 2.49GB  ...nts/optim_075000_rank7.pt: 1%| | 17.4MB / 2.49GB  ...nts/optim_075000_rank6.pt: 1%| | 26.8MB / 2.49GB  Processing Files (0 / 7) : 1%| | 216MB / 22.2GB, 30.8MB/s New Data Upload : 16%|█▋ | 131MB / 805MB, 18.8MB/s  ...nts/optim_075000_rank0.pt: 1%| | 18.7MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 70.8MB / 2.49GB  ...nts/optim_075000_rank3.pt: 1%|▏ | 31.5MB / 2.49GB  ...nts/optim_075000_rank4.pt: 1%| | 24.9MB / 2.49GB  ...nts/optim_075000_rank5.pt: 2%|▏ | 42.0MB / 2.49GB  ...nts/optim_075000_rank7.pt: 1%| | 22.3MB / 2.49GB  ...nts/optim_075000_rank6.pt: 1%|▏ | 32.6MB / 2.49GB  Processing Files (0 / 7) : 1%| | 243MB / 22.2GB, 33.7MB/s New Data Upload : 17%|█▋ | 145MB / 872MB, 20.2MB/s  ...nts/optim_075000_rank0.pt: 1%| | 21.8MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 72.8MB / 2.49GB  ...nts/optim_075000_rank3.pt: 2%|▏ | 39.8MB / 2.49GB  ...nts/optim_075000_rank4.pt: 1%| | 27.2MB / 2.49GB  ...nts/optim_075000_rank5.pt: 2%|▏ | 46.6MB / 2.49GB  ...nts/optim_075000_rank7.pt: 1%| | 28.2MB / 2.49GB  ...nts/optim_075000_rank6.pt: 2%|▏ | 38.1MB / 2.49GB  Processing Files (0 / 7) : 1%| | 275MB / 22.2GB, 37.1MB/s New Data Upload : 19%|█▊ | 163MB / 872MB, 22.0MB/s  ...nts/optim_075000_rank0.pt: 1%| | 27.5MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 76.1MB / 2.49GB  ...nts/optim_075000_rank3.pt: 2%|▏ | 46.6MB / 2.49GB  ...nts/optim_075000_rank4.pt: 1%|▏ | 31.2MB / 2.49GB  ...nts/optim_075000_rank5.pt: 2%|▏ | 52.0MB / 2.49GB  ...nts/optim_075000_rank7.pt: 2%|▏ | 38.3MB / 2.49GB  ...nts/optim_075000_rank6.pt: 2%|▏ | 44.9MB / 2.49GB  Processing Files (0 / 7) : 1%|▏ | 317MB / 22.2GB, 41.7MB/s New Data Upload : 20%|█▉ | 185MB / 939MB, 24.4MB/s  ...nts/optim_075000_rank0.pt: 1%| | 30.7MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 80.2MB / 2.49GB  ...nts/optim_075000_rank3.pt: 2%|▏ | 51.9MB / 2.49GB  ...nts/optim_075000_rank4.pt: 2%|▏ | 38.8MB / 2.49GB  ...nts/optim_075000_rank5.pt: 2%|▏ | 58.9MB / 2.49GB  ...nts/optim_075000_rank7.pt: 2%|▏ | 49.4MB / 2.49GB  ...nts/optim_075000_rank6.pt: 2%|▏ | 52.5MB / 2.49GB  Processing Files (0 / 7) : 2%|▏ | 363MB / 22.2GB, 46.5MB/s New Data Upload : 22%|██▏ | 207MB / 939MB, 26.6MB/s  ...nts/optim_075000_rank0.pt: 1%|▏ | 33.3MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 81.6MB / 2.49GB  ...nts/optim_075000_rank3.pt: 2%|▏ | 56.5MB / 2.49GB  ...nts/optim_075000_rank4.pt: 2%|▏ | 42.8MB / 2.49GB  ...nts/optim_075000_rank5.pt: 3%|▎ | 62.5MB / 2.49GB  ...nts/optim_075000_rank7.pt: 2%|▏ | 54.0MB / 2.49GB  ...nts/optim_075000_rank6.pt: 2%|▏ | 59.8MB / 2.49GB  Processing Files (0 / 7) : 2%|▏ | 391MB / 22.2GB, 48.8MB/s New Data Upload : 24%|██▎ | 221MB / 939MB, 27.7MB/s  ...eckpoints/model_075000.pt: 0%| | 3.66MB / 2.35GB  ...nts/optim_075000_rank0.pt: 2%|▏ | 37.7MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 85.5MB / 2.49GB  ...nts/optim_075000_rank3.pt: 3%|▎ | 62.1MB / 2.49GB  ...nts/optim_075000_rank4.pt: 2%|▏ | 49.8MB / 2.49GB  ...nts/optim_075000_rank5.pt: 3%|▎ | 66.1MB / 2.49GB  ...nts/optim_075000_rank7.pt: 3%|▎ | 63.6MB / 2.49GB  ...nts/optim_075000_rank6.pt: 3%|▎ | 67.1MB / 2.49GB  ...eckpoints/model_075000.pt: 0%| | 3.66MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 436MB / 22.2GB, 53.1MB/s New Data Upload : 23%|██▎ | 242MB / 1.07GB, 29.5MB/s  ...nts/optim_075000_rank0.pt: 2%|▏ | 44.2MB / 2.49GB  ...nts/optim_075000_rank1.pt: 4%|▎ | 88.3MB / 2.49GB  ...nts/optim_075000_rank3.pt: 3%|▎ | 66.2MB / 2.49GB  ...nts/optim_075000_rank4.pt: 2%|▏ | 56.2MB / 2.49GB  ...nts/optim_075000_rank5.pt: 3%|▎ | 70.7MB / 2.49GB  ...nts/optim_075000_rank7.pt: 3%|▎ | 70.7MB / 2.49GB  ...nts/optim_075000_rank6.pt: 3%|▎ | 74.0MB / 2.49GB  ...eckpoints/model_075000.pt: 0%| | 3.66MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 474MB / 22.2GB, 56.4MB/s New Data Upload : 24%|██▍ | 263MB / 1.07GB, 31.3MB/s  ...nts/optim_075000_rank0.pt: 2%|▏ | 53.5MB / 2.49GB  ...nts/optim_075000_rank1.pt: 4%|▍ | 95.3MB / 2.49GB  ...nts/optim_075000_rank3.pt: 3%|▎ | 73.6MB / 2.49GB  ...nts/optim_075000_rank4.pt: 3%|▎ | 65.9MB / 2.49GB  ...nts/optim_075000_rank5.pt: 3%|▎ | 79.1MB / 2.49GB  ...nts/optim_075000_rank7.pt: 3%|▎ | 84.3MB / 2.49GB  ...nts/optim_075000_rank6.pt: 3%|▎ | 84.3MB / 2.49GB  ...eckpoints/model_075000.pt: 0%| | 4.83MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 541MB / 22.2GB, 62.9MB/s New Data Upload : 27%|██▋ | 291MB / 1.07GB, 33.9MB/s  ...nts/optim_075000_rank0.pt: 2%|▏ | 59.3MB / 2.49GB  ...nts/optim_075000_rank1.pt: 4%|▍ | 96.2MB / 2.49GB  ...nts/optim_075000_rank3.pt: 3%|▎ | 76.5MB / 2.49GB  ...nts/optim_075000_rank4.pt: 3%|▎ | 69.7MB / 2.49GB  ...nts/optim_075000_rank5.pt: 3%|▎ | 82.0MB / 2.49GB  ...nts/optim_075000_rank7.pt: 3%|▎ | 86.5MB / 2.49GB  ...nts/optim_075000_rank6.pt: 4%|▎ | 89.3MB / 2.49GB  ...eckpoints/model_075000.pt: 0%| | 8.32MB / 2.35GB  Processing Files (0 / 8) : 3%|▎ | 568MB / 22.2GB, 64.5MB/s New Data Upload : 29%|██▉ | 309MB / 1.07GB, 35.1MB/s  ...nts/optim_075000_rank0.pt: 3%|▎ | 70.8MB / 2.49GB  ...nts/optim_075000_rank1.pt: 4%|▍ | 100MB / 2.49GB  ...nts/optim_075000_rank3.pt: 3%|▎ | 81.5MB / 2.49GB  ...nts/optim_075000_rank4.pt: 3%|▎ | 77.6MB / 2.49GB  ...nts/optim_075000_rank5.pt: 4%|▎ | 87.9MB / 2.49GB  ...nts/optim_075000_rank7.pt: 4%|▍ | 97.7MB / 2.49GB  ...nts/optim_075000_rank6.pt: 4%|▍ | 98.6MB / 2.49GB  ...eckpoints/model_075000.pt: 1%| | 11.8MB / 2.35GB  Processing Files (0 / 8) : 3%|▎ | 626MB / 22.2GB, 69.6MB/s New Data Upload : 30%|██▉ | 342MB / 1.14GB, 38.0MB/s  ...nts/optim_075000_rank0.pt: 3%|▎ | 79.2MB / 2.49GB  ...nts/optim_075000_rank1.pt: 4%|▍ | 106MB / 2.49GB  ...nts/optim_075000_rank3.pt: 4%|▎ | 88.0MB / 2.49GB  ...nts/optim_075000_rank4.pt: 3%|▎ | 85.9MB / 2.49GB  ...nts/optim_075000_rank5.pt: 4%|▎ | 93.1MB / 2.49GB  ...nts/optim_075000_rank7.pt: 4%|▍ | 106MB / 2.49GB  ...nts/optim_075000_rank6.pt: 4%|▍ | 110MB / 2.49GB  ...eckpoints/model_075000.pt: 1%| | 18.2MB / 2.35GB  Processing Files (0 / 8) : 3%|▎ | 686MB / 22.2GB, 74.6MB/s New Data Upload : 33%|███▎ | 371MB / 1.14GB, 40.3MB/s  ...nts/optim_075000_rank0.pt: 4%|▎ | 92.3MB / 2.49GB  ...nts/optim_075000_rank1.pt: 5%|▍ | 114MB / 2.49GB  ...nts/optim_075000_rank3.pt: 4%|▍ | 96.2MB / 2.49GB  ...nts/optim_075000_rank4.pt: 4%|▍ | 95.8MB / 2.49GB  ...nts/optim_075000_rank5.pt: 4%|▍ | 101MB / 2.49GB  ...nts/optim_075000_rank7.pt: 5%|▍ | 117MB / 2.49GB  ...nts/optim_075000_rank6.pt: 5%|▍ | 118MB / 2.49GB  ...eckpoints/model_075000.pt: 1%| | 26.4MB / 2.35GB  Processing Files (0 / 8) : 3%|▎ | 760MB / 22.2GB, 80.9MB/s New Data Upload : 35%|███▌ | 405MB / 1.14GB, 43.0MB/s  ...nts/optim_075000_rank0.pt: 4%|▍ | 105MB / 2.49GB  ...nts/optim_075000_rank1.pt: 5%|▍ | 121MB / 2.49GB  ...nts/optim_075000_rank3.pt: 4%|▍ | 104MB / 2.49GB  ...nts/optim_075000_rank4.pt: 4%|▍ | 105MB / 2.49GB  ...nts/optim_075000_rank5.pt: 4%|▍ | 109MB / 2.49GB  ...nts/optim_075000_rank7.pt: 5%|▍ | 124MB / 2.49GB  ...nts/optim_075000_rank6.pt: 5%|▌ | 124MB / 2.49GB  ...eckpoints/model_075000.pt: 1%|▏ | 30.4MB / 2.35GB  Processing Files (0 / 8) : 4%|▎ | 822MB / 22.2GB, 85.6MB/s New Data Upload : 36%|███▌ | 431MB / 1.21GB, 44.9MB/s  ...nts/optim_075000_rank0.pt: 5%|▍ | 120MB / 2.49GB  ...nts/optim_075000_rank1.pt: 5%|▌ | 131MB / 2.49GB  ...nts/optim_075000_rank3.pt: 5%|▍ | 115MB / 2.49GB  ...nts/optim_075000_rank4.pt: 5%|▍ | 116MB / 2.49GB  ...nts/optim_075000_rank5.pt: 5%|▍ | 121MB / 2.49GB  ...nts/optim_075000_rank7.pt: 5%|▌ | 134MB / 2.49GB  ...nts/optim_075000_rank6.pt: 5%|▌ | 135MB / 2.49GB  ...eckpoints/model_075000.pt: 2%|▏ | 39.7MB / 2.35GB  Processing Files (0 / 8) : 4%|▍ | 911MB / 22.2GB, 92.9MB/s New Data Upload : 38%|███▊ | 464MB / 1.21GB, 47.3MB/s  ...nts/optim_075000_rank0.pt: 5%|▌ | 130MB / 2.49GB  ...nts/optim_075000_rank1.pt: 5%|▌ | 137MB / 2.49GB  ...nts/optim_075000_rank3.pt: 5%|▍ | 119MB / 2.49GB  ...nts/optim_075000_rank4.pt: 5%|▍ | 124MB / 2.49GB  ...nts/optim_075000_rank5.pt: 5%|▌ | 127MB / 2.49GB  ...nts/optim_075000_rank7.pt: 6%|▌ | 138MB / 2.49GB  ...nts/optim_075000_rank6.pt: 6%|▌ | 143MB / 2.49GB  ...eckpoints/model_075000.pt: 2%|▏ | 43.2MB / 2.35GB  Processing Files (0 / 8) : 4%|▍ | 962MB / 22.2GB, 96.2MB/s New Data Upload : 40%|████ | 487MB / 1.21GB, 48.7MB/s  ...nts/optim_075000_rank0.pt: 6%|▌ | 140MB / 2.49GB  ...nts/optim_075000_rank1.pt: 6%|▌ | 145MB / 2.49GB  ...nts/optim_075000_rank3.pt: 5%|▌ | 128MB / 2.49GB  ...nts/optim_075000_rank4.pt: 5%|▌ | 135MB / 2.49GB  ...nts/optim_075000_rank5.pt: 5%|▌ | 136MB / 2.49GB  ...nts/optim_075000_rank7.pt: 6%|▌ | 145MB / 2.49GB  ...nts/optim_075000_rank6.pt: 6%|▋ | 156MB / 2.49GB  ...eckpoints/model_075000.pt: 2%|▏ | 51.4MB / 2.35GB  Processing Files (0 / 8) : 5%|▍ | 1.04GB / 22.2GB, 102MB/s New Data Upload : 43%|████▎ | 519MB / 1.21GB, 50.8MB/s  ...nts/optim_075000_rank0.pt: 6%|▋ | 156MB / 2.49GB  ...nts/optim_075000_rank1.pt: 6%|▋ | 157MB / 2.49GB  ...nts/optim_075000_rank3.pt: 6%|▌ | 142MB / 2.49GB  ...nts/optim_075000_rank4.pt: 6%|▌ | 150MB / 2.49GB  ...nts/optim_075000_rank5.pt: 6%|▌ | 147MB / 2.49GB  ...nts/optim_075000_rank7.pt: 6%|▋ | 156MB / 2.49GB  ...nts/optim_075000_rank6.pt: 7%|▋ | 173MB / 2.49GB  ...eckpoints/model_075000.pt: 3%|▎ | 60.1MB / 2.35GB  Processing Files (0 / 8) : 5%|▌ | 1.14GB / 22.2GB, 112MB/s New Data Upload : 44%|████▎ | 556MB / 1.27GB, 54.6MB/s  ...nts/optim_075000_rank0.pt: 7%|▋ | 172MB / 2.49GB  ...nts/optim_075000_rank1.pt: 7%|▋ | 175MB / 2.49GB  ...nts/optim_075000_rank3.pt: 6%|▋ | 160MB / 2.49GB  ...nts/optim_075000_rank4.pt: 7%|▋ | 167MB / 2.49GB  ...nts/optim_075000_rank5.pt: 7%|▋ | 166MB / 2.49GB  ...nts/optim_075000_rank7.pt: 7%|▋ | 173MB / 2.49GB  ...nts/optim_075000_rank6.pt: 8%|▊ | 190MB / 2.49GB  ...eckpoints/model_075000.pt: 3%|▎ | 66.5MB / 2.35GB  Processing Files (0 / 8) : 6%|▌ | 1.27GB / 22.2GB, 124MB/s New Data Upload : 46%|████▋ | 590MB / 1.27GB, 57.8MB/s  ...nts/optim_075000_rank0.pt: 7%|▋ | 179MB / 2.49GB  ...nts/optim_075000_rank1.pt: 7%|▋ | 183MB / 2.49GB  ...nts/optim_075000_rank3.pt: 7%|▋ | 169MB / 2.49GB  ...nts/optim_075000_rank4.pt: 7%|▋ | 177MB / 2.49GB  ...nts/optim_075000_rank5.pt: 7%|▋ | 174MB / 2.49GB  ...nts/optim_075000_rank7.pt: 7%|▋ | 180MB / 2.49GB  ...nts/optim_075000_rank6.pt: 8%|▊ | 199MB / 2.49GB  ...eckpoints/model_075000.pt: 3%|▎ | 70.0MB / 2.35GB  Processing Files (0 / 8) : 6%|▌ | 1.33GB / 22.2GB, 130MB/s New Data Upload : 45%|████▌ | 609MB / 1.34GB, 59.7MB/s  ...nts/optim_075000_rank0.pt: 7%|▋ | 185MB / 2.49GB  ...nts/optim_075000_rank1.pt: 8%|▊ | 191MB / 2.49GB  ...nts/optim_075000_rank3.pt: 7%|▋ | 178MB / 2.49GB  ...nts/optim_075000_rank4.pt: 8%|▊ | 187MB / 2.49GB  ...nts/optim_075000_rank5.pt: 7%|▋ | 183MB / 2.49GB  ...nts/optim_075000_rank7.pt: 8%|▊ | 188MB / 2.49GB  ...nts/optim_075000_rank6.pt: 8%|▊ | 207MB / 2.49GB  ...eckpoints/model_075000.pt: 3%|▎ | 70.0MB / 2.35GB  Processing Files (0 / 8) : 6%|▌ | 1.39GB / 22.2GB, 136MB/s New Data Upload : 46%|████▋ | 623MB / 1.34GB, 61.0MB/s  ...nts/optim_075000_rank0.pt: 8%|▊ | 205MB / 2.49GB  ...nts/optim_075000_rank1.pt: 9%|▊ | 211MB / 2.49GB  ...nts/optim_075000_rank3.pt: 8%|▊ | 199MB / 2.49GB  ...nts/optim_075000_rank4.pt: 8%|▊ | 207MB / 2.49GB  ...nts/optim_075000_rank5.pt: 8%|▊ | 206MB / 2.49GB  ...nts/optim_075000_rank7.pt: 8%|▊ | 208MB / 2.49GB  ...nts/optim_075000_rank6.pt: 9%|▉ | 225MB / 2.49GB  ...eckpoints/model_075000.pt: 3%|▎ | 70.0MB / 2.35GB  Processing Files (0 / 8) : 7%|▋ | 1.53GB / 22.2GB, 150MB/s New Data Upload : 46%|████▋ | 653MB / 1.41GB, 63.9MB/s  ...nts/optim_075000_rank0.pt: 9%|▉ | 221MB / 2.49GB  ...nts/optim_075000_rank1.pt: 9%|▉ | 228MB / 2.49GB  ...nts/optim_075000_rank3.pt: 9%|▊ | 215MB / 2.49GB  ...nts/optim_075000_rank4.pt: 9%|▉ | 223MB / 2.49GB  ...nts/optim_075000_rank5.pt: 9%|▉ | 224MB / 2.49GB  ...nts/optim_075000_rank7.pt: 9%|▉ | 225MB / 2.49GB  ...nts/optim_075000_rank6.pt: 10%|▉ | 243MB / 2.49GB  ...eckpoints/model_075000.pt: 3%|▎ | 70.0MB / 2.35GB  Processing Files (0 / 8) : 7%|▋ | 1.65GB / 22.2GB, 161MB/s New Data Upload : 46%|████▌ | 676MB / 1.48GB, 66.1MB/s  ...nts/optim_075000_rank0.pt: 10%|▉ | 239MB / 2.49GB  ...nts/optim_075000_rank1.pt: 10%|▉ | 246MB / 2.49GB  ...nts/optim_075000_rank3.pt: 9%|▉ | 234MB / 2.49GB  ...nts/optim_075000_rank4.pt: 10%|▉ | 240MB / 2.49GB  ...nts/optim_075000_rank5.pt: 10%|▉ | 241MB / 2.49GB  ...nts/optim_075000_rank7.pt: 10%|▉ | 242MB / 2.49GB  ...nts/optim_075000_rank6.pt: 10%|█ | 260MB / 2.49GB  ...eckpoints/model_075000.pt: 3%|▎ | 74.0MB / 2.35GB  Processing Files (0 / 8) : 8%|▊ | 1.78GB / 22.2GB, 173MB/s New Data Upload : 46%|████▌ | 704MB / 1.54GB, 68.5MB/s  ...nts/optim_075000_rank0.pt: 11%|█ | 262MB / 2.49GB  ...nts/optim_075000_rank1.pt: 11%|█ | 269MB / 2.49GB  ...nts/optim_075000_rank3.pt: 10%|█ | 258MB / 2.49GB  ...nts/optim_075000_rank4.pt: 11%|█ | 264MB / 2.49GB  ...nts/optim_075000_rank5.pt: 11%|█ | 265MB / 2.49GB  ...nts/optim_075000_rank7.pt: 11%|█ | 266MB / 2.49GB  ...nts/optim_075000_rank6.pt: 11%|█▏ | 282MB / 2.49GB  ...eckpoints/model_075000.pt: 3%|▎ | 78.6MB / 2.35GB  Processing Files (0 / 8) : 9%|▊ | 1.94GB / 22.2GB, 189MB/s New Data Upload : 46%|████▌ | 739MB / 1.61GB, 71.6MB/s  ...nts/optim_075000_rank0.pt: 11%|█▏ | 284MB / 2.49GB  ...nts/optim_075000_rank1.pt: 12%|█▏ | 290MB / 2.49GB  ...nts/optim_075000_rank3.pt: 11%|█▏ | 280MB / 2.49GB  ...nts/optim_075000_rank4.pt: 11%|█▏ | 285MB / 2.49GB  ...nts/optim_075000_rank5.pt: 12%|█▏ | 287MB / 2.49GB  ...nts/optim_075000_rank7.pt: 12%|█▏ | 288MB / 2.49GB  ...nts/optim_075000_rank6.pt: 12%|█▏ | 300MB / 2.49GB  ...eckpoints/model_075000.pt: 3%|▎ | 80.9MB / 2.35GB  Processing Files (0 / 8) : 9%|▉ | 2.09GB / 22.2GB, 204MB/s New Data Upload : 46%|████▌ | 768MB / 1.68GB, 74.3MB/s  ...nts/optim_075000_rank0.pt: 13%|█▎ | 318MB / 2.49GB  ...nts/optim_075000_rank1.pt: 13%|█▎ | 323MB / 2.49GB  ...nts/optim_075000_rank3.pt: 13%|█▎ | 315MB / 2.49GB  ...nts/optim_075000_rank4.pt: 13%|█▎ | 320MB / 2.49GB  ...nts/optim_075000_rank5.pt: 13%|█▎ | 321MB / 2.49GB  ...nts/optim_075000_rank7.pt: 13%|█▎ | 321MB / 2.49GB  ...nts/optim_075000_rank6.pt: 13%|█▎ | 332MB / 2.49GB  ...eckpoints/model_075000.pt: 4%|▎ | 84.8MB / 2.35GB  Processing Files (0 / 8) : 11%|█ | 2.34GB / 22.2GB, 226MB/s New Data Upload : 47%|████▋ | 813MB / 1.74GB, 78.0MB/s  ...nts/optim_075000_rank0.pt: 14%|█▍ | 352MB / 2.49GB  ...nts/optim_075000_rank1.pt: 14%|█▍ | 357MB / 2.49GB  ...nts/optim_075000_rank3.pt: 14%|█▍ | 349MB / 2.49GB  ...nts/optim_075000_rank4.pt: 14%|█▍ | 352MB / 2.49GB  ...nts/optim_075000_rank5.pt: 14%|█▍ | 356MB / 2.49GB  ...nts/optim_075000_rank7.pt: 14%|█▍ | 356MB / 2.49GB  ...nts/optim_075000_rank6.pt: 15%|█▍ | 368MB / 2.49GB  ...eckpoints/model_075000.pt: 4%|▎ | 86.5MB / 2.35GB  Processing Files (0 / 8) : 12%|█▏ | 2.58GB / 22.2GB, 250MB/s New Data Upload : 49%|████▉ | 857MB / 1.74GB, 81.9MB/s  ...nts/optim_075000_rank0.pt: 15%|█▌ | 379MB / 2.49GB  ...nts/optim_075000_rank1.pt: 15%|█▌ | 384MB / 2.49GB  ...nts/optim_075000_rank3.pt: 15%|█▌ | 373MB / 2.49GB  ...nts/optim_075000_rank4.pt: 15%|█▌ | 378MB / 2.49GB  ...nts/optim_075000_rank5.pt: 15%|█▌ | 384MB / 2.49GB  ...nts/optim_075000_rank7.pt: 15%|█▌ | 384MB / 2.49GB  ...nts/optim_075000_rank6.pt: 16%|█▌ | 393MB / 2.49GB  ...eckpoints/model_075000.pt: 4%|▍ | 89.9MB / 2.35GB  Processing Files (0 / 8) : 12%|█▏ | 2.77GB / 22.2GB, 268MB/s New Data Upload : 51%|█████ | 892MB / 1.74GB, 85.2MB/s  ...nts/optim_075000_rank0.pt: 17%|█▋ | 412MB / 2.49GB  ...nts/optim_075000_rank1.pt: 17%|█▋ | 417MB / 2.49GB  ...nts/optim_075000_rank3.pt: 16%|█▋ | 407MB / 2.49GB  ...nts/optim_075000_rank4.pt: 17%|█▋ | 411MB / 2.49GB  ...nts/optim_075000_rank5.pt: 17%|█▋ | 417MB / 2.49GB  ...nts/optim_075000_rank7.pt: 17%|█▋ | 417MB / 2.49GB  ...nts/optim_075000_rank6.pt: 17%|█▋ | 426MB / 2.49GB  ...eckpoints/model_075000.pt: 4%|▍ | 93.9MB / 2.35GB  Processing Files (0 / 8) : 13%|█▎ | 3.00GB / 22.2GB, 290MB/s New Data Upload : 50%|████▉ | 937MB / 1.88GB, 88.9MB/s  ...nts/optim_075000_rank0.pt: 18%|█▊ | 445MB / 2.49GB  ...nts/optim_075000_rank1.pt: 18%|█▊ | 449MB / 2.49GB  ...nts/optim_075000_rank3.pt: 18%|█▊ | 441MB / 2.49GB  ...nts/optim_075000_rank4.pt: 18%|█▊ | 444MB / 2.49GB  ...nts/optim_075000_rank5.pt: 18%|█▊ | 450MB / 2.49GB  ...nts/optim_075000_rank7.pt: 18%|█▊ | 451MB / 2.49GB  ...nts/optim_075000_rank6.pt: 18%|█▊ | 456MB / 2.49GB  ...eckpoints/model_075000.pt: 4%|▍ | 97.8MB / 2.35GB  Processing Files (0 / 8) : 15%|█▍ | 3.23GB / 22.2GB, 312MB/s New Data Upload : 50%|█████ | 980MB / 1.94GB, 92.7MB/s  ...nts/optim_075000_rank0.pt: 19%|█▉ | 466MB / 2.49GB  ...nts/optim_075000_rank1.pt: 19%|█▉ | 470MB / 2.49GB  ...nts/optim_075000_rank3.pt: 19%|█▊ | 461MB / 2.49GB  ...nts/optim_075000_rank4.pt: 19%|█▊ | 464MB / 2.49GB  ...nts/optim_075000_rank5.pt: 19%|█▉ | 471MB / 2.49GB  ...nts/optim_075000_rank7.pt: 19%|█▉ | 472MB / 2.49GB  ...nts/optim_075000_rank6.pt: 19%|█▉ | 476MB / 2.49GB  ...eckpoints/model_075000.pt: 4%|▍ | 102MB / 2.35GB  Processing Files (0 / 8) : 15%|█▌ | 3.38GB / 22.2GB, 326MB/s New Data Upload : 50%|█████ | 1.01GB / 2.01GB, 95.1MB/s  ...nts/optim_075000_rank0.pt: 20%|█▉ | 497MB / 2.49GB  ...nts/optim_075000_rank1.pt: 20%|██ | 500MB / 2.49GB  ...nts/optim_075000_rank3.pt: 20%|█▉ | 491MB / 2.49GB  ...nts/optim_075000_rank4.pt: 20%|█▉ | 495MB / 2.49GB  ...nts/optim_075000_rank5.pt: 20%|██ | 502MB / 2.49GB  ...nts/optim_075000_rank7.pt: 20%|██ | 503MB / 2.49GB  ...nts/optim_075000_rank6.pt: 20%|██ | 503MB / 2.49GB  ...eckpoints/model_075000.pt: 4%|▍ | 105MB / 2.35GB  Processing Files (0 / 8) : 16%|█▌ | 3.60GB / 22.2GB, 347MB/s New Data Upload : 52%|█████▏ | 1.05GB / 2.01GB, 98.7MB/s  ...nts/optim_075000_rank0.pt: 21%|██ | 517MB / 2.49GB  ...nts/optim_075000_rank1.pt: 21%|██ | 519MB / 2.49GB  ...nts/optim_075000_rank3.pt: 21%|██ | 510MB / 2.49GB  ...nts/optim_075000_rank4.pt: 21%|██ | 514MB / 2.49GB  ...nts/optim_075000_rank5.pt: 21%|██ | 522MB / 2.49GB  ...nts/optim_075000_rank7.pt: 21%|██ | 522MB / 2.49GB  ...nts/optim_075000_rank6.pt: 21%|██ | 520MB / 2.49GB  ...eckpoints/model_075000.pt: 5%|▍ | 112MB / 2.35GB  Processing Files (0 / 8) : 17%|█▋ | 3.74GB / 22.2GB, 360MB/s New Data Upload : 52%|█████▏ | 1.08GB / 2.08GB, 101MB/s  ...nts/optim_075000_rank0.pt: 22%|██▏ | 543MB / 2.49GB  ...nts/optim_075000_rank1.pt: 22%|██▏ | 544MB / 2.49GB  ...nts/optim_075000_rank3.pt: 22%|██▏ | 535MB / 2.49GB  ...nts/optim_075000_rank4.pt: 22%|██▏ | 540MB / 2.49GB  ...nts/optim_075000_rank5.pt: 22%|██▏ | 547MB / 2.49GB  ...nts/optim_075000_rank7.pt: 22%|██▏ | 547MB / 2.49GB  ...nts/optim_075000_rank6.pt: 22%|██▏ | 546MB / 2.49GB  ...eckpoints/model_075000.pt: 5%|▍ | 114MB / 2.35GB  Processing Files (0 / 8) : 18%|█▊ | 3.92GB / 22.2GB, 377MB/s New Data Upload : 50%|█████ | 1.11GB / 2.21GB, 104MB/s  ...nts/optim_075000_rank0.pt: 23%|██▎ | 574MB / 2.49GB  ...nts/optim_075000_rank1.pt: 23%|██▎ | 577MB / 2.49GB  ...nts/optim_075000_rank3.pt: 23%|██▎ | 566MB / 2.49GB  ...nts/optim_075000_rank4.pt: 23%|██▎ | 571MB / 2.49GB  ...nts/optim_075000_rank5.pt: 23%|██▎ | 580MB / 2.49GB  ...nts/optim_075000_rank7.pt: 23%|██▎ | 578MB / 2.49GB  ...nts/optim_075000_rank6.pt: 23%|██▎ | 577MB / 2.49GB  ...eckpoints/model_075000.pt: 5%|▌ | 121MB / 2.35GB  Processing Files (0 / 8) : 19%|█▊ | 4.14GB / 22.2GB, 398MB/s New Data Upload : 52%|█████▏ | 1.15GB / 2.21GB, 108MB/s  ...nts/optim_075000_rank0.pt: 24%|██▍ | 595MB / 2.49GB  ...nts/optim_075000_rank1.pt: 24%|██▍ | 599MB / 2.49GB  ...nts/optim_075000_rank3.pt: 24%|██▎ | 587MB / 2.49GB  ...nts/optim_075000_rank4.pt: 24%|██▍ | 591MB / 2.49GB  ...nts/optim_075000_rank5.pt: 24%|██▍ | 601MB / 2.49GB  ...nts/optim_075000_rank7.pt: 24%|██▍ | 597MB / 2.49GB  ...nts/optim_075000_rank6.pt: 24%|██▍ | 597MB / 2.49GB  ...eckpoints/model_075000.pt: 5%|▌ | 126MB / 2.35GB  Processing Files (0 / 8) : 19%|█▉ | 4.29GB / 22.2GB, 412MB/s New Data Upload : 50%|█████ | 1.18GB / 2.35GB, 110MB/s  ...nts/optim_075000_rank0.pt: 25%|██▍ | 619MB / 2.49GB  ...nts/optim_075000_rank1.pt: 25%|██▌ | 625MB / 2.49GB  ...nts/optim_075000_rank3.pt: 25%|██▍ | 609MB / 2.49GB  ...nts/optim_075000_rank4.pt: 25%|██▍ | 613MB / 2.49GB  ...nts/optim_075000_rank5.pt: 25%|██▌ | 629MB / 2.49GB  ...nts/optim_075000_rank7.pt: 25%|██▍ | 620MB / 2.49GB  ...nts/optim_075000_rank6.pt: 25%|██▍ | 620MB / 2.49GB  ...eckpoints/model_075000.pt: 5%|▌ | 127MB / 2.35GB  Processing Files (0 / 8) : 20%|██ | 4.46GB / 22.2GB, 429MB/s New Data Upload : 52%|█████▏ | 1.22GB / 2.35GB, 113MB/s  ...nts/optim_075000_rank0.pt: 26%|██▋ | 653MB / 2.49GB  ...nts/optim_075000_rank1.pt: 27%|██▋ | 660MB / 2.49GB  ...nts/optim_075000_rank3.pt: 26%|██▌ | 642MB / 2.49GB  ...nts/optim_075000_rank4.pt: 26%|██▌ | 647MB / 2.49GB  ...nts/optim_075000_rank5.pt: 27%|██▋ | 662MB / 2.49GB  ...nts/optim_075000_rank7.pt: 26%|██▋ | 654MB / 2.49GB  ...nts/optim_075000_rank6.pt: 26%|██▋ | 654MB / 2.49GB  ...eckpoints/model_075000.pt: 6%|▌ | 135MB / 2.35GB  Processing Files (0 / 8) : 21%|██ | 4.71GB / 22.2GB, 452MB/s New Data Upload : 53%|█████▎ | 1.27GB / 2.41GB, 118MB/s  ...nts/optim_075000_rank0.pt: 27%|██▋ | 676MB / 2.49GB  ...nts/optim_075000_rank1.pt: 27%|██▋ | 683MB / 2.49GB  ...nts/optim_075000_rank3.pt: 27%|██▋ | 665MB / 2.49GB  ...nts/optim_075000_rank4.pt: 27%|██▋ | 670MB / 2.49GB  ...nts/optim_075000_rank5.pt: 28%|██▊ | 686MB / 2.49GB  ...nts/optim_075000_rank7.pt: 27%|██▋ | 676MB / 2.49GB  ...nts/optim_075000_rank6.pt: 27%|██▋ | 676MB / 2.49GB  ...eckpoints/model_075000.pt: 6%|▌ | 137MB / 2.35GB  Processing Files (0 / 8) : 22%|██▏ | 4.87GB / 22.2GB, 468MB/s New Data Upload : 52%|█████▏ | 1.30GB / 2.48GB, 121MB/s  ...nts/optim_075000_rank0.pt: 28%|██▊ | 706MB / 2.49GB  ...nts/optim_075000_rank1.pt: 29%|██▊ | 714MB / 2.49GB  ...nts/optim_075000_rank3.pt: 28%|██▊ | 697MB / 2.49GB  ...nts/optim_075000_rank4.pt: 28%|██▊ | 702MB / 2.49GB  ...nts/optim_075000_rank5.pt: 29%|██▉ | 715MB / 2.49GB  ...nts/optim_075000_rank7.pt: 28%|██▊ | 706MB / 2.49GB  ...nts/optim_075000_rank6.pt: 28%|██▊ | 707MB / 2.49GB  ...eckpoints/model_075000.pt: 6%|▋ | 147MB / 2.35GB  Processing Files (0 / 8) : 23%|██▎ | 5.10GB / 22.2GB, 490MB/s New Data Upload : 54%|█████▍ | 1.35GB / 2.48GB, 126MB/s  ...nts/optim_075000_rank0.pt: 29%|██▉ | 730MB / 2.49GB  ...nts/optim_075000_rank1.pt: 30%|██▉ | 740MB / 2.49GB  ...nts/optim_075000_rank3.pt: 29%|██▉ | 722MB / 2.49GB  ...nts/optim_075000_rank4.pt: 29%|██▉ | 728MB / 2.49GB  ...nts/optim_075000_rank5.pt: 30%|██▉ | 741MB / 2.49GB  ...nts/optim_075000_rank7.pt: 29%|██▉ | 729MB / 2.49GB  ...nts/optim_075000_rank6.pt: 29%|██▉ | 732MB / 2.49GB  ...eckpoints/model_075000.pt: 6%|▋ | 150MB / 2.35GB  Processing Files (0 / 8) : 24%|██▎ | 5.27GB / 22.2GB, 507MB/s New Data Upload : 53%|█████▎ | 1.39GB / 2.62GB, 129MB/s  ...nts/optim_075000_rank0.pt: 31%|███ | 758MB / 2.49GB  ...nts/optim_075000_rank1.pt: 31%|███ | 769MB / 2.49GB  ...nts/optim_075000_rank3.pt: 30%|███ | 750MB / 2.49GB  ...nts/optim_075000_rank4.pt: 30%|███ | 756MB / 2.49GB  ...nts/optim_075000_rank5.pt: 31%|███ | 770MB / 2.49GB  ...nts/optim_075000_rank7.pt: 30%|███ | 757MB / 2.49GB  ...nts/optim_075000_rank6.pt: 31%|███ | 759MB / 2.49GB  ...eckpoints/model_075000.pt: 7%|▋ | 157MB / 2.35GB  Processing Files (0 / 8) : 25%|██▍ | 5.48GB / 22.2GB, 527MB/s New Data Upload : 55%|█████▍ | 1.43GB / 2.62GB, 134MB/s  ...nts/optim_075000_rank0.pt: 31%|███▏ | 779MB / 2.49GB  ...nts/optim_075000_rank1.pt: 32%|███▏ | 793MB / 2.49GB  ...nts/optim_075000_rank3.pt: 31%|███ | 774MB / 2.49GB  ...nts/optim_075000_rank4.pt: 31%|███▏ | 780MB / 2.49GB  ...nts/optim_075000_rank5.pt: 32%|███▏ | 792MB / 2.49GB  ...nts/optim_075000_rank7.pt: 31%|███▏ | 778MB / 2.49GB  ...nts/optim_075000_rank6.pt: 31%|███▏ | 782MB / 2.49GB  ...eckpoints/model_075000.pt: 7%|▋ | 162MB / 2.35GB  Processing Files (0 / 8) : 25%|██▌ | 5.64GB / 22.2GB, 542MB/s New Data Upload : 55%|█████▍ | 1.47GB / 2.68GB, 137MB/s  ...nts/optim_075000_rank0.pt: 32%|███▏ | 797MB / 2.49GB  ...nts/optim_075000_rank1.pt: 33%|███▎ | 814MB / 2.49GB  ...nts/optim_075000_rank3.pt: 32%|███▏ | 793MB / 2.49GB  ...nts/optim_075000_rank4.pt: 32%|███▏ | 799MB / 2.49GB  ...nts/optim_075000_rank5.pt: 33%|███▎ | 811MB / 2.49GB  ...nts/optim_075000_rank7.pt: 32%|███▏ | 796MB / 2.49GB  ...nts/optim_075000_rank6.pt: 32%|███▏ | 801MB / 2.49GB  ...eckpoints/model_075000.pt: 7%|▋ | 171MB / 2.35GB  Processing Files (0 / 8) : 26%|██▌ | 5.78GB / 22.2GB, 556MB/s New Data Upload : 55%|█████▍ | 1.50GB / 2.75GB, 140MB/s  ...nts/optim_075000_rank0.pt: 33%|███▎ | 821MB / 2.49GB  ...nts/optim_075000_rank1.pt: 34%|███▍ | 841MB / 2.49GB  ...nts/optim_075000_rank3.pt: 33%|███▎ | 817MB / 2.49GB  ...nts/optim_075000_rank4.pt: 33%|███▎ | 823MB / 2.49GB  ...nts/optim_075000_rank5.pt: 34%|███▎ | 835MB / 2.49GB  ...nts/optim_075000_rank7.pt: 33%|███▎ | 820MB / 2.49GB  ...nts/optim_075000_rank6.pt: 33%|███▎ | 825MB / 2.49GB  ...eckpoints/model_075000.pt: 8%|▊ | 181MB / 2.35GB  Processing Files (0 / 8) : 27%|██▋ | 5.96GB / 22.2GB, 573MB/s New Data Upload : 56%|█████▌ | 1.55GB / 2.75GB, 144MB/s  ...nts/optim_075000_rank0.pt: 34%|███▍ | 846MB / 2.49GB  ...nts/optim_075000_rank1.pt: 35%|███▍ | 870MB / 2.49GB  ...nts/optim_075000_rank3.pt: 34%|███▍ | 844MB / 2.49GB  ...nts/optim_075000_rank4.pt: 34%|███▍ | 850MB / 2.49GB  ...nts/optim_075000_rank5.pt: 35%|███▍ | 859MB / 2.49GB  ...nts/optim_075000_rank7.pt: 34%|███▍ | 845MB / 2.49GB  ...nts/optim_075000_rank6.pt: 34%|███▍ | 850MB / 2.49GB  ...eckpoints/model_075000.pt: 8%|▊ | 183MB / 2.35GB  Processing Files (0 / 8) : 28%|██▊ | 6.15GB / 22.2GB, 590MB/s New Data Upload : 56%|█████▌ | 1.58GB / 2.82GB, 147MB/s  ...nts/optim_075000_rank0.pt: 35%|███▌ | 872MB / 2.49GB  ...nts/optim_075000_rank1.pt: 36%|███▌ | 899MB / 2.49GB  ...nts/optim_075000_rank3.pt: 35%|███▌ | 871MB / 2.49GB  ...nts/optim_075000_rank4.pt: 35%|███▌ | 876MB / 2.49GB  ...nts/optim_075000_rank5.pt: 36%|███▌ | 887MB / 2.49GB  ...nts/optim_075000_rank7.pt: 35%|███▌ | 871MB / 2.49GB  ...nts/optim_075000_rank6.pt: 35%|███▌ | 876MB / 2.49GB  ...eckpoints/model_075000.pt: 8%|▊ | 197MB / 2.35GB  Processing Files (0 / 8) : 29%|██▊ | 6.35GB / 22.2GB, 609MB/s New Data Upload : 58%|█████▊ | 1.63GB / 2.82GB, 151MB/s  ...nts/optim_075000_rank0.pt: 36%|███▌ | 897MB / 2.49GB  ...nts/optim_075000_rank1.pt: 37%|███▋ | 928MB / 2.49GB  ...nts/optim_075000_rank3.pt: 36%|███▌ | 897MB / 2.49GB  ...nts/optim_075000_rank4.pt: 36%|███▋ | 902MB / 2.49GB  ...nts/optim_075000_rank5.pt: 37%|███▋ | 913MB / 2.49GB  ...nts/optim_075000_rank7.pt: 36%|███▌ | 896MB / 2.49GB  ...nts/optim_075000_rank6.pt: 36%|███▋ | 902MB / 2.49GB  ...eckpoints/model_075000.pt: 9%|▉ | 206MB / 2.35GB  Processing Files (0 / 8) : 29%|██▉ | 6.54GB / 22.2GB, 625MB/s New Data Upload : 60%|█████▉ | 1.68GB / 2.82GB, 155MB/s  ...nts/optim_075000_rank0.pt: 37%|███▋ | 918MB / 2.49GB  ...nts/optim_075000_rank1.pt: 38%|███▊ | 950MB / 2.49GB  ...nts/optim_075000_rank3.pt: 37%|███▋ | 918MB / 2.49GB  ...nts/optim_075000_rank4.pt: 37%|███▋ | 926MB / 2.49GB  ...nts/optim_075000_rank5.pt: 38%|███▊ | 939MB / 2.49GB  ...nts/optim_075000_rank7.pt: 37%|███▋ | 918MB / 2.49GB  ...nts/optim_075000_rank6.pt: 37%|███▋ | 924MB / 2.49GB  ...eckpoints/model_075000.pt: 9%|▉ | 211MB / 2.35GB  Processing Files (0 / 8) : 30%|███ | 6.70GB / 22.2GB, 640MB/s New Data Upload : 58%|█████▊ | 1.71GB / 2.95GB, 157MB/s  ...nts/optim_075000_rank0.pt: 38%|███▊ | 943MB / 2.49GB  ...nts/optim_075000_rank1.pt: 39%|███▉ | 979MB / 2.49GB  ...nts/optim_075000_rank3.pt: 38%|███▊ | 943MB / 2.49GB  ...nts/optim_075000_rank4.pt: 38%|███▊ | 953MB / 2.49GB  ...nts/optim_075000_rank5.pt: 39%|███▉ | 966MB / 2.49GB  ...nts/optim_075000_rank7.pt: 38%|███▊ | 946MB / 2.49GB  ...nts/optim_075000_rank6.pt: 38%|███▊ | 950MB / 2.49GB  ...eckpoints/model_075000.pt: 9%|▉ | 219MB / 2.35GB  Processing Files (0 / 8) : 31%|███ | 6.90GB / 22.2GB, 657MB/s New Data Upload : 60%|█████▉ | 1.76GB / 2.95GB, 160MB/s  ...nts/optim_075000_rank0.pt: 39%|███▉ | 969MB / 2.49GB  ...nts/optim_075000_rank1.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_075000_rank3.pt: 39%|███▉ | 971MB / 2.49GB  ...nts/optim_075000_rank4.pt: 39%|███▉ | 980MB / 2.49GB  ...nts/optim_075000_rank5.pt: 40%|███▉ | 993MB / 2.49GB  ...nts/optim_075000_rank7.pt: 39%|███▉ | 974MB / 2.49GB  ...nts/optim_075000_rank6.pt: 39%|███▉ | 976MB / 2.49GB  ...eckpoints/model_075000.pt: 10%|▉ | 224MB / 2.35GB  Processing Files (0 / 8) : 32%|███▏ | 7.10GB / 22.2GB, 675MB/s New Data Upload : 60%|█████▉ | 1.80GB / 3.02GB, 164MB/s  ...nts/optim_075000_rank0.pt: 40%|███▉ | 987MB / 2.49GB  ...nts/optim_075000_rank1.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_075000_rank3.pt: 40%|███▉ | 990MB / 2.49GB  ...nts/optim_075000_rank4.pt: 40%|████ | 1.00GB / 2.49GB  ...nts/optim_075000_rank5.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_075000_rank7.pt: 40%|███▉ | 991MB / 2.49GB  ...nts/optim_075000_rank6.pt: 40%|████ | 996MB / 2.49GB  ...eckpoints/model_075000.pt: 10%|▉ | 231MB / 2.35GB  Processing Files (0 / 8) : 33%|███▎ | 7.23GB / 22.2GB, 685MB/s New Data Upload : 59%|█████▉ | 1.83GB / 3.08GB, 165MB/s  ...nts/optim_075000_rank0.pt: 40%|████ | 1.00GB / 2.49GB  ...nts/optim_075000_rank1.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_075000_rank3.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_075000_rank4.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_075000_rank5.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_075000_rank7.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_075000_rank6.pt: 41%|████ | 1.01GB / 2.49GB  ...eckpoints/model_075000.pt: 10%|█ | 236MB / 2.35GB  Processing Files (0 / 8) : 33%|███▎ | 7.35GB / 22.2GB, 694MB/s New Data Upload : 60%|██████ | 1.86GB / 3.08GB, 166MB/s  ...nts/optim_075000_rank0.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_075000_rank1.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_075000_rank3.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_075000_rank4.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_075000_rank5.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_075000_rank7.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_075000_rank6.pt: 42%|████▏ | 1.04GB / 2.49GB  ...eckpoints/model_075000.pt: 10%|█ | 238MB / 2.35GB  Processing Files (0 / 8) : 34%|███▍ | 7.52GB / 22.2GB, 707MB/s New Data Upload : 61%|██████▏ | 1.90GB / 3.08GB, 168MB/s  ...nts/optim_075000_rank0.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_075000_rank1.pt: 43%|████▎ | 1.08GB / 2.49GB  ...nts/optim_075000_rank3.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_075000_rank4.pt: 42%|████▏ | 1.06GB / 2.49GB  ...nts/optim_075000_rank5.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_075000_rank7.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_075000_rank6.pt: 42%|████▏ | 1.05GB / 2.49GB  ...eckpoints/model_075000.pt: 10%|█ | 243MB / 2.35GB  Processing Files (0 / 8) : 34%|███▍ | 7.62GB / 22.2GB, 712MB/s New Data Upload : 62%|██████▏ | 1.92GB / 3.08GB, 168MB/s  ...nts/optim_075000_rank0.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_075000_rank1.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_075000_rank3.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_075000_rank4.pt: 43%|████▎ | 1.08GB / 2.49GB  ...nts/optim_075000_rank5.pt: 44%|████▎ | 1.09GB / 2.49GB  ...nts/optim_075000_rank7.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_075000_rank6.pt: 43%|████▎ | 1.07GB / 2.49GB  ...eckpoints/model_075000.pt: 10%|█ | 245MB / 2.35GB  Processing Files (0 / 8) : 35%|███▌ | 7.79GB / 22.2GB, 725MB/s New Data Upload : 63%|██████▎ | 1.95GB / 3.08GB, 170MB/s  ...nts/optim_075000_rank0.pt: 43%|████▎ | 1.08GB / 2.49GB  ...nts/optim_075000_rank1.pt: 45%|████▍ | 1.12GB / 2.49GB  ...nts/optim_075000_rank3.pt: 44%|████▎ | 1.08GB / 2.49GB  ...nts/optim_075000_rank4.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_075000_rank5.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_075000_rank7.pt: 44%|████▍ | 1.09GB / 2.49GB  ...nts/optim_075000_rank6.pt: 44%|████▎ | 1.09GB / 2.49GB  ...eckpoints/model_075000.pt: 11%|█ | 248MB / 2.35GB  Processing Files (0 / 8) : 36%|███▌ | 7.90GB / 22.2GB, 732MB/s New Data Upload : 64%|██████▍ | 1.98GB / 3.08GB, 171MB/s  ...nts/optim_075000_rank0.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_075000_rank1.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_075000_rank3.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_075000_rank4.pt: 45%|████▌ | 1.12GB / 2.49GB  ...nts/optim_075000_rank5.pt: 45%|████▌ | 1.13GB / 2.49GB  ...nts/optim_075000_rank7.pt: 45%|████▌ | 1.12GB / 2.49GB  ...nts/optim_075000_rank6.pt: 45%|████▍ | 1.11GB / 2.49GB  ...eckpoints/model_075000.pt: 11%|█ | 253MB / 2.35GB  Processing Files (0 / 8) : 36%|███▋ | 8.09GB / 22.2GB, 747MB/s New Data Upload : 64%|██████▍ | 2.02GB / 3.15GB, 173MB/s  ...nts/optim_075000_rank0.pt: 45%|████▌ | 1.13GB / 2.49GB  ...nts/optim_075000_rank1.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_075000_rank3.pt: 46%|████▌ | 1.13GB / 2.49GB  ...nts/optim_075000_rank4.pt: 46%|████▌ | 1.15GB / 2.49GB  ...nts/optim_075000_rank5.pt: 46%|████▋ | 1.15GB / 2.49GB  ...nts/optim_075000_rank7.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_075000_rank6.pt: 46%|████▌ | 1.14GB / 2.49GB  ...eckpoints/model_075000.pt: 11%|█ | 257MB / 2.35GB  Processing Files (0 / 8) : 37%|███▋ | 8.26GB / 22.2GB, 757MB/s New Data Upload : 63%|██████▎ | 2.06GB / 3.29GB, 173MB/s  ...nts/optim_075000_rank0.pt: 46%|████▋ | 1.15GB / 2.49GB  ...nts/optim_075000_rank1.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_075000_rank3.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_075000_rank4.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_075000_rank5.pt: 47%|████▋ | 1.18GB / 2.49GB  ...nts/optim_075000_rank7.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_075000_rank6.pt: 47%|████▋ | 1.16GB / 2.49GB  ...eckpoints/model_075000.pt: 11%|█ | 259MB / 2.35GB  Processing Files (0 / 8) : 38%|███▊ | 8.45GB / 22.2GB, 773MB/s New Data Upload : 61%|██████▏ | 2.10GB / 3.42GB, 175MB/s  ...nts/optim_075000_rank0.pt: 47%|████▋ | 1.18GB / 2.49GB  ...nts/optim_075000_rank1.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_075000_rank3.pt: 48%|████▊ | 1.18GB / 2.49GB  ...nts/optim_075000_rank4.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_075000_rank5.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_075000_rank7.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_075000_rank6.pt: 48%|████▊ | 1.19GB / 2.49GB  ...eckpoints/model_075000.pt: 11%|█ | 261MB / 2.35GB  Processing Files (0 / 8) : 39%|███▉ | 8.62GB / 22.2GB, 784MB/s New Data Upload : 60%|█████▉ | 2.13GB / 3.55GB, 175MB/s  ...nts/optim_075000_rank0.pt: 49%|████▊ | 1.21GB / 2.49GB  ...nts/optim_075000_rank1.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_075000_rank3.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_075000_rank4.pt: 50%|████▉ | 1.23GB / 2.49GB  ...nts/optim_075000_rank5.pt: 50%|████▉ | 1.23GB / 2.49GB  ...nts/optim_075000_rank7.pt: 49%|████▉ | 1.23GB / 2.49GB  ...nts/optim_075000_rank6.pt: 49%|████▉ | 1.22GB / 2.49GB  ...eckpoints/model_075000.pt: 11%|█▏ | 267MB / 2.35GB  Processing Files (0 / 8) : 40%|███▉ | 8.86GB / 22.2GB, 801MB/s New Data Upload : 60%|██████ | 2.18GB / 3.62GB, 178MB/s  ...nts/optim_075000_rank0.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_075000_rank1.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_075000_rank3.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_075000_rank4.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_075000_rank5.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_075000_rank7.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_075000_rank6.pt: 50%|█████ | 1.25GB / 2.49GB  ...eckpoints/model_075000.pt: 12%|█▏ | 273MB / 2.35GB  Processing Files (0 / 8) : 41%|████ | 9.08GB / 22.2GB, 816MB/s New Data Upload : 59%|█████▉ | 2.23GB / 3.76GB, 179MB/s  ...nts/optim_075000_rank0.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_075000_rank1.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_075000_rank3.pt: 52%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_075000_rank4.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_075000_rank5.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_075000_rank7.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_075000_rank6.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...eckpoints/model_075000.pt: 12%|█▏ | 279MB / 2.35GB  Processing Files (0 / 8) : 42%|████▏ | 9.35GB / 22.2GB, 836MB/s New Data Upload : 60%|█████▉ | 2.28GB / 3.82GB, 182MB/s  ...nts/optim_075000_rank0.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_075000_rank1.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_075000_rank3.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_075000_rank4.pt: 54%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_075000_rank5.pt: 54%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_075000_rank7.pt: 54%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_075000_rank6.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...eckpoints/model_075000.pt: 12%|█▏ | 289MB / 2.35GB  Processing Files (0 / 8) : 43%|████▎ | 9.58GB / 22.2GB, 850MB/s New Data Upload : 60%|██████ | 2.34GB / 3.89GB, 184MB/s  ...nts/optim_075000_rank0.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_075000_rank1.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_075000_rank3.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_075000_rank4.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_075000_rank5.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_075000_rank7.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_075000_rank6.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...eckpoints/model_075000.pt: 13%|█▎ | 299MB / 2.35GB  Processing Files (0 / 8) : 44%|████▍ | 9.83GB / 22.2GB, 870MB/s New Data Upload : 62%|██████▏ | 2.39GB / 3.89GB, 187MB/s  ...nts/optim_075000_rank0.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_075000_rank1.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_075000_rank3.pt: 56%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_075000_rank4.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_075000_rank5.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_075000_rank7.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_075000_rank6.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...eckpoints/model_075000.pt: 13%|█▎ | 309MB / 2.35GB  Processing Files (0 / 8) : 45%|████▌ | 10.0GB / 22.2GB, 883MB/s New Data Upload : 63%|██████▎ | 2.45GB / 3.89GB, 189MB/s  ...nts/optim_075000_rank0.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_075000_rank1.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_075000_rank3.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...nts/optim_075000_rank4.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_075000_rank5.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_075000_rank7.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_075000_rank6.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...eckpoints/model_075000.pt: 14%|█▎ | 321MB / 2.35GB  Processing Files (0 / 8) : 46%|████▌ | 10.2GB / 22.2GB, 892MB/s New Data Upload : 64%|██████▍ | 2.49GB / 3.89GB, 190MB/s  ...nts/optim_075000_rank0.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_075000_rank1.pt: 59%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_075000_rank4.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_075000_rank5.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_075000_rank7.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_075000_rank6.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...eckpoints/model_075000.pt: 14%|█▍ | 326MB / 2.35GB  Processing Files (0 / 8) : 47%|████▋ | 10.5GB / 22.2GB, 900MB/s New Data Upload : 64%|██████▍ | 2.54GB / 3.96GB, 191MB/s  ...nts/optim_075000_rank0.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_075000_rank1.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_075000_rank3.pt: 59%|█████▉ | 1.46GB / 2.49GB  ...nts/optim_075000_rank4.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_075000_rank5.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_075000_rank7.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_075000_rank6.pt: 59%|█████▉ | 1.46GB / 2.49GB  ...eckpoints/model_075000.pt: 14%|█▍ | 336MB / 2.35GB  Processing Files (0 / 8) : 48%|████▊ | 10.6GB / 22.2GB, 913MB/s New Data Upload : 65%|██████▌ | 2.58GB / 3.96GB, 194MB/s  ...nts/optim_075000_rank0.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_075000_rank1.pt: 61%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_075000_rank3.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_075000_rank5.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_075000_rank7.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_075000_rank6.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...eckpoints/model_075000.pt: 15%|█▍ | 345MB / 2.35GB  Processing Files (0 / 8) : 49%|████▊ | 10.8GB / 22.2GB, 926MB/s New Data Upload : 65%|██████▌ | 2.63GB / 4.02GB, 196MB/s  ...nts/optim_075000_rank0.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_075000_rank1.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_075000_rank3.pt: 61%|██████ | 1.52GB / 2.49GB  ...nts/optim_075000_rank4.pt: 61%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_075000_rank5.pt: 61%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_075000_rank7.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_075000_rank6.pt: 61%|██████ | 1.52GB / 2.49GB  ...eckpoints/model_075000.pt: 15%|█▌ | 357MB / 2.35GB  Processing Files (0 / 8) : 50%|████▉ | 11.0GB / 22.2GB, 933MB/s New Data Upload : 66%|██████▋ | 2.67GB / 4.02GB, 198MB/s  ...nts/optim_075000_rank0.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_075000_rank1.pt: 63%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_075000_rank3.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_075000_rank4.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_075000_rank5.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_075000_rank7.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_075000_rank6.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...eckpoints/model_075000.pt: 16%|█▌ | 366MB / 2.35GB  Processing Files (0 / 8) : 50%|█████ | 11.2GB / 22.2GB, 939MB/s New Data Upload : 66%|██████▋ | 2.72GB / 4.09GB, 200MB/s  ...nts/optim_075000_rank0.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_075000_rank1.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_075000_rank3.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_075000_rank4.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_075000_rank5.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_075000_rank7.pt: 63%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_075000_rank6.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...eckpoints/model_075000.pt: 16%|█▌ | 377MB / 2.35GB  Processing Files (0 / 8) : 51%|█████ | 11.4GB / 22.2GB, 940MB/s New Data Upload : 66%|██████▋ | 2.75GB / 4.16GB, 201MB/s  ...nts/optim_075000_rank0.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_075000_rank1.pt: 65%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_075000_rank3.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_075000_rank4.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_075000_rank5.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_075000_rank7.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_075000_rank6.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...eckpoints/model_075000.pt: 16%|█▋ | 385MB / 2.35GB  Processing Files (0 / 8) : 52%|█████▏ | 11.6GB / 22.2GB, 944MB/s New Data Upload : 67%|██████▋ | 2.80GB / 4.16GB, 203MB/s  ...nts/optim_075000_rank0.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_075000_rank1.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_075000_rank3.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_075000_rank4.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_075000_rank5.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_075000_rank7.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_075000_rank6.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...eckpoints/model_075000.pt: 17%|█▋ | 394MB / 2.35GB  Processing Files (0 / 8) : 53%|█████▎ | 11.8GB / 22.2GB, 949MB/s New Data Upload : 69%|██████▊ | 2.85GB / 4.16GB, 204MB/s  ...nts/optim_075000_rank0.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_075000_rank1.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_075000_rank3.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_075000_rank4.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_075000_rank5.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_075000_rank7.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_075000_rank6.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...eckpoints/model_075000.pt: 17%|█▋ | 403MB / 2.35GB  Processing Files (0 / 8) : 54%|█████▎ | 11.9GB / 22.2GB, 941MB/s New Data Upload : 67%|██████▋ | 2.89GB / 4.29GB, 204MB/s  ...nts/optim_075000_rank0.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_075000_rank1.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_075000_rank3.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_075000_rank4.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_075000_rank5.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_075000_rank7.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_075000_rank6.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...eckpoints/model_075000.pt: 18%|█▊ | 412MB / 2.35GB  Processing Files (0 / 8) : 55%|█████▍ | 12.1GB / 22.2GB, 936MB/s New Data Upload : 68%|██████▊ | 2.94GB / 4.29GB, 204MB/s  ...nts/optim_075000_rank0.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_075000_rank1.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_075000_rank3.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_075000_rank4.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_075000_rank5.pt: 67%|██████▋ | 1.68GB / 2.49GB  ...nts/optim_075000_rank7.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_075000_rank6.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...eckpoints/model_075000.pt: 18%|█▊ | 421MB / 2.35GB  Processing Files (0 / 8) : 55%|█████▌ | 12.2GB / 22.2GB, 930MB/s New Data Upload : 69%|██████▉ | 2.97GB / 4.29GB, 204MB/s  ...nts/optim_075000_rank0.pt: 69%|██████▊ | 1.71GB / 2.49GB  ...nts/optim_075000_rank1.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_075000_rank3.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_075000_rank4.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_075000_rank5.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_075000_rank7.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_075000_rank6.pt: 69%|██████▊ | 1.71GB / 2.49GB  ...eckpoints/model_075000.pt: 18%|█▊ | 427MB / 2.35GB  Processing Files (0 / 8) : 56%|█████▌ | 12.4GB / 22.2GB, 923MB/s New Data Upload : 70%|███████ | 3.01GB / 4.29GB, 204MB/s  ...nts/optim_075000_rank2.pt: 3%|▎ | 72.0MB / 2.49GB  ...nts/optim_075000_rank0.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_075000_rank1.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_075000_rank3.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_075000_rank4.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_075000_rank5.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_075000_rank7.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_075000_rank6.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...eckpoints/model_075000.pt: 18%|█▊ | 432MB / 2.35GB  ...nts/optim_075000_rank2.pt: 3%|▎ | 72.0MB / 2.49GB  Processing Files (0 / 9) : 57%|█████▋ | 12.7GB / 22.2GB, 924MB/s New Data Upload : 71%|███████ | 3.05GB / 4.29GB, 203MB/s  ...nts/optim_075000_rank0.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_075000_rank1.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_075000_rank3.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_075000_rank4.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_075000_rank5.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_075000_rank7.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_075000_rank6.pt: 71%|███████ | 1.75GB / 2.49GB  ...eckpoints/model_075000.pt: 19%|█▊ | 439MB / 2.35GB  ...nts/optim_075000_rank2.pt: 10%|█ | 256MB / 2.49GB  Processing Files (0 / 9) : 58%|█████▊ | 13.0GB / 22.2GB, 942MB/s New Data Upload : 72%|███████▏ | 3.09GB / 4.29GB, 204MB/s  ...nts/optim_075000_rank0.pt: 71%|███████▏ | 1.77GB / 2.49GB  ...nts/optim_075000_rank1.pt: 73%|███████▎ | 1.80GB / 2.49GB  ...nts/optim_075000_rank3.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_075000_rank4.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_075000_rank5.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_075000_rank7.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_075000_rank6.pt: 71%|███████▏ | 1.77GB / 2.49GB  ...eckpoints/model_075000.pt: 19%|█▉ | 444MB / 2.35GB  ...nts/optim_075000_rank2.pt: 15%|█▍ | 365MB / 2.49GB  Processing Files (0 / 9) : 60%|█████▉ | 13.2GB / 22.2GB, 946MB/s New Data Upload : 71%|███████ | 3.13GB / 4.42GB, 204MB/s  ...nts/optim_075000_rank0.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_075000_rank1.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_075000_rank3.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_075000_rank4.pt: 73%|███████▎ | 1.80GB / 2.49GB  ...nts/optim_075000_rank5.pt: 71%|███████▏ | 1.77GB / 2.49GB  ...nts/optim_075000_rank7.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_075000_rank6.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...eckpoints/model_075000.pt: 19%|█▉ | 448MB / 2.35GB  ...nts/optim_075000_rank2.pt: 22%|██▏ | 557MB / 2.49GB  Processing Files (0 / 9) : 61%|██████ | 13.6GB / 22.2GB, 964MB/s New Data Upload : 70%|███████ | 3.16GB / 4.48GB, 204MB/s  ...nts/optim_075000_rank0.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_075000_rank1.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_075000_rank3.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_075000_rank4.pt: 74%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_075000_rank5.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_075000_rank7.pt: 73%|███████▎ | 1.80GB / 2.49GB  ...nts/optim_075000_rank6.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...eckpoints/model_075000.pt: 19%|█▉ | 458MB / 2.35GB  ...nts/optim_075000_rank2.pt: 30%|██▉ | 734MB / 2.49GB  Processing Files (0 / 9) : 63%|██████▎ | 13.9GB / 22.2GB, 981MB/s New Data Upload : 72%|███████▏ | 3.21GB / 4.48GB, 206MB/s  ...nts/optim_075000_rank0.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_075000_rank1.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_075000_rank3.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_075000_rank4.pt: 75%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_075000_rank5.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_075000_rank7.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_075000_rank6.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...eckpoints/model_075000.pt: 20%|█▉ | 462MB / 2.35GB  ...nts/optim_075000_rank2.pt: 34%|███▍ | 856MB / 2.49GB  Processing Files (0 / 9) : 64%|██████▍ | 14.2GB / 22.2GB, 988MB/s New Data Upload : 71%|███████▏ | 3.25GB / 4.55GB, 206MB/s  ...nts/optim_075000_rank0.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_075000_rank1.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_075000_rank3.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_075000_rank4.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_075000_rank5.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_075000_rank7.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_075000_rank6.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...eckpoints/model_075000.pt: 20%|██ | 474MB / 2.35GB  ...nts/optim_075000_rank2.pt: 36%|███▋ | 903MB / 2.49GB  Processing Files (0 / 9) : 65%|██████▍ | 14.4GB / 22.2GB, 991MB/s New Data Upload : 71%|███████▏ | 3.29GB / 4.62GB, 207MB/s  ...nts/optim_075000_rank0.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_075000_rank1.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_075000_rank3.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_075000_rank4.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_075000_rank5.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_075000_rank7.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_075000_rank6.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...eckpoints/model_075000.pt: 21%|██ | 485MB / 2.35GB  ...nts/optim_075000_rank2.pt: 42%|████▏ | 1.04GB / 2.49GB  Processing Files (0 / 9) : 66%|██████▌ | 14.7GB / 22.2GB, 1.00GB/s New Data Upload : 72%|███████▏ | 3.34GB / 4.62GB, 208MB/s  ...nts/optim_075000_rank0.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_075000_rank1.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_075000_rank3.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_075000_rank4.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_075000_rank5.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_075000_rank7.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_075000_rank6.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...eckpoints/model_075000.pt: 21%|██ | 489MB / 2.35GB  ...nts/optim_075000_rank2.pt: 47%|████▋ | 1.18GB / 2.49GB  Processing Files (0 / 9) : 67%|██████▋ | 15.0GB / 22.2GB, 1.01GB/s New Data Upload : 72%|███████▏ | 3.38GB / 4.69GB, 207MB/s  ...nts/optim_075000_rank0.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_075000_rank1.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_075000_rank3.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_075000_rank4.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_075000_rank5.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_075000_rank7.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_075000_rank6.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...eckpoints/model_075000.pt: 21%|██▏ | 504MB / 2.35GB  ...nts/optim_075000_rank2.pt: 52%|█████▏ | 1.29GB / 2.49GB  Processing Files (0 / 9) : 68%|██████▊ | 15.2GB / 22.2GB, 1.01GB/s New Data Upload : 73%|███████▎ | 3.42GB / 4.69GB, 208MB/s  ...nts/optim_075000_rank0.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_075000_rank1.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_075000_rank3.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_075000_rank4.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_075000_rank5.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_075000_rank7.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_075000_rank6.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...eckpoints/model_075000.pt: 22%|██▏ | 519MB / 2.35GB  ...nts/optim_075000_rank2.pt: 57%|█████▋ | 1.43GB / 2.49GB  Processing Files (0 / 9) : 70%|██████▉ | 15.5GB / 22.2GB, 1.02GB/s New Data Upload : 73%|███████▎ | 3.47GB / 4.75GB, 208MB/s  ...nts/optim_075000_rank0.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_075000_rank1.pt: 80%|███████▉ | 1.99GB / 2.49GB  ...nts/optim_075000_rank3.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_075000_rank4.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_075000_rank5.pt: 77%|███████▋ | 1.93GB / 2.49GB  ...nts/optim_075000_rank7.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_075000_rank6.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...eckpoints/model_075000.pt: 23%|██▎ | 533MB / 2.35GB  ...nts/optim_075000_rank2.pt: 59%|█████▉ | 1.48GB / 2.49GB  Processing Files (0 / 9) : 71%|███████ | 15.7GB / 22.2GB, 1.02GB/s New Data Upload : 74%|███████▍ | 3.51GB / 4.75GB, 208MB/s  ...nts/optim_075000_rank0.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_075000_rank1.pt: 81%|████████ | 2.00GB / 2.49GB  ...nts/optim_075000_rank3.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_075000_rank4.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_075000_rank5.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_075000_rank7.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_075000_rank6.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...eckpoints/model_075000.pt: 23%|██▎ | 547MB / 2.35GB  ...nts/optim_075000_rank2.pt: 63%|██████▎ | 1.58GB / 2.49GB  Processing Files (0 / 9) : 72%|███████▏ | 15.9GB / 22.2GB, 1.02GB/s New Data Upload : 74%|███████▎ | 3.54GB / 4.82GB, 207MB/s  ...nts/optim_075000_rank0.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_075000_rank1.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_075000_rank3.pt: 80%|███████▉ | 1.99GB / 2.49GB  ...nts/optim_075000_rank4.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_075000_rank5.pt: 79%|███████▊ | 1.96GB / 2.49GB  ...nts/optim_075000_rank7.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_075000_rank6.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...eckpoints/model_075000.pt: 24%|██▍ | 561MB / 2.35GB  ...nts/optim_075000_rank2.pt: 68%|██████▊ | 1.69GB / 2.49GB  Processing Files (0 / 9) : 73%|███████▎ | 16.1GB / 22.2GB, 1.03GB/s New Data Upload : 74%|███████▍ | 3.59GB / 4.82GB, 208MB/s  ...nts/optim_075000_rank0.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_075000_rank1.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_075000_rank3.pt: 81%|████████ | 2.00GB / 2.49GB  ...nts/optim_075000_rank4.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_075000_rank5.pt: 79%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_075000_rank7.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_075000_rank6.pt: 81%|████████ | 2.00GB / 2.49GB  ...eckpoints/model_075000.pt: 25%|██▍ | 579MB / 2.35GB  ...nts/optim_075000_rank2.pt: 76%|███████▌ | 1.88GB / 2.49GB  Processing Files (0 / 9) : 74%|███████▍ | 16.5GB / 22.2GB, 1.05GB/s New Data Upload : 74%|███████▍ | 3.63GB / 4.89GB, 209MB/s  ...nts/optim_075000_rank0.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_075000_rank1.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_075000_rank3.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_075000_rank4.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_075000_rank5.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_075000_rank7.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_075000_rank6.pt: 81%|████████ | 2.02GB / 2.49GB  ...eckpoints/model_075000.pt: 26%|██▌ | 600MB / 2.35GB  ...nts/optim_075000_rank2.pt: 78%|███████▊ | 1.94GB / 2.49GB  Processing Files (0 / 9) : 75%|███████▍ | 16.7GB / 22.2GB, 1.05GB/s New Data Upload : 74%|███████▍ | 3.68GB / 4.95GB, 209MB/s  ...nts/optim_075000_rank0.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_075000_rank1.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_075000_rank3.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_075000_rank4.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_075000_rank5.pt: 81%|████████ | 2.00GB / 2.49GB  ...nts/optim_075000_rank7.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_075000_rank6.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...eckpoints/model_075000.pt: 26%|██▌ | 616MB / 2.35GB  ...nts/optim_075000_rank2.pt: 82%|████████▏ | 2.03GB / 2.49GB  Processing Files (0 / 9) : 76%|███████▌ | 16.9GB / 22.2GB, 1.05GB/s New Data Upload : 74%|███████▍ | 3.71GB / 5.02GB, 209MB/s  ...nts/optim_075000_rank0.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_075000_rank1.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_075000_rank3.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_075000_rank4.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_075000_rank5.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_075000_rank7.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_075000_rank6.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...eckpoints/model_075000.pt: 27%|██▋ | 631MB / 2.35GB  ...nts/optim_075000_rank2.pt: 82%|████████▏ | 2.04GB / 2.49GB  Processing Files (0 / 9) : 76%|███████▋ | 17.0GB / 22.2GB, 1.04GB/s New Data Upload : 75%|███████▍ | 3.75GB / 5.02GB, 208MB/s  ...nts/optim_075000_rank0.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_075000_rank1.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_075000_rank3.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_075000_rank4.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_075000_rank5.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_075000_rank7.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_075000_rank6.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...eckpoints/model_075000.pt: 28%|██▊ | 652MB / 2.35GB  ...nts/optim_075000_rank2.pt: 83%|████████▎ | 2.06GB / 2.49GB  Processing Files (0 / 9) : 77%|███████▋ | 17.1GB / 22.2GB, 1.04GB/s New Data Upload : 75%|███████▍ | 3.80GB / 5.09GB, 208MB/s  ...nts/optim_075000_rank0.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_075000_rank1.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_075000_rank3.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_075000_rank4.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_075000_rank5.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_075000_rank7.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_075000_rank6.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...eckpoints/model_075000.pt: 29%|██▊ | 671MB / 2.35GB  ...nts/optim_075000_rank2.pt: 83%|████████▎ | 2.07GB / 2.49GB  Processing Files (0 / 9) : 78%|███████▊ | 17.3GB / 22.2GB, 1.04GB/s New Data Upload : 75%|███████▍ | 3.84GB / 5.16GB, 209MB/s  ...nts/optim_075000_rank0.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_075000_rank1.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_075000_rank3.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_075000_rank4.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_075000_rank5.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_075000_rank7.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_075000_rank6.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...eckpoints/model_075000.pt: 30%|██▉ | 693MB / 2.35GB  ...nts/optim_075000_rank2.pt: 84%|████████▍ | 2.09GB / 2.49GB  Processing Files (0 / 9) : 78%|███████▊ | 17.4GB / 22.2GB, 1.03GB/s New Data Upload : 74%|███████▍ | 3.88GB / 5.22GB, 208MB/s  ...nts/optim_075000_rank0.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_075000_rank1.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_075000_rank3.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_075000_rank4.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_075000_rank5.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_075000_rank7.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_075000_rank6.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...eckpoints/model_075000.pt: 30%|███ | 715MB / 2.35GB  ...nts/optim_075000_rank2.pt: 85%|████████▍ | 2.10GB / 2.49GB  Processing Files (0 / 9) : 79%|███████▉ | 17.5GB / 22.2GB, 1.02GB/s New Data Upload : 74%|███████▍ | 3.93GB / 5.29GB, 209MB/s  ...nts/optim_075000_rank0.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_075000_rank1.pt: 86%|████████▋ | 2.14GB / 2.49GB  ...nts/optim_075000_rank3.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_075000_rank4.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_075000_rank5.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_075000_rank7.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_075000_rank6.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...eckpoints/model_075000.pt: 31%|███ | 732MB / 2.35GB  ...nts/optim_075000_rank2.pt: 85%|████████▍ | 2.11GB / 2.49GB  Processing Files (0 / 9) : 79%|███████▉ | 17.6GB / 22.2GB, 1.02GB/s New Data Upload : 74%|███████▍ | 3.96GB / 5.36GB, 209MB/s  ...nts/optim_075000_rank0.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_075000_rank1.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_075000_rank3.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_075000_rank4.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_075000_rank5.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_075000_rank7.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_075000_rank6.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...eckpoints/model_075000.pt: 32%|███▏ | 758MB / 2.35GB  ...nts/optim_075000_rank2.pt: 85%|████████▌ | 2.12GB / 2.49GB  Processing Files (0 / 9) : 80%|███████▉ | 17.8GB / 22.2GB, 1.02GB/s New Data Upload : 74%|███████▍ | 4.00GB / 5.42GB, 210MB/s  ...nts/optim_075000_rank0.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_075000_rank1.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_075000_rank3.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_075000_rank4.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_075000_rank5.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_075000_rank7.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_075000_rank6.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...eckpoints/model_075000.pt: 33%|███▎ | 778MB / 2.35GB  ...nts/optim_075000_rank2.pt: 86%|████████▌ | 2.14GB / 2.49GB  Processing Files (0 / 9) : 80%|████████ | 17.9GB / 22.2GB, 1.01GB/s New Data Upload : 74%|███████▍ | 4.04GB / 5.42GB, 210MB/s  ...nts/optim_075000_rank0.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_075000_rank1.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_075000_rank3.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_075000_rank4.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_075000_rank5.pt: 85%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_075000_rank7.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_075000_rank6.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...eckpoints/model_075000.pt: 34%|███▍ | 802MB / 2.35GB  ...nts/optim_075000_rank2.pt: 86%|████████▋ | 2.15GB / 2.49GB  Processing Files (0 / 9) : 81%|████████ | 18.0GB / 22.2GB, 1.02GB/s New Data Upload : 74%|███████▍ | 4.08GB / 5.49GB, 212MB/s  ...nts/optim_075000_rank0.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_075000_rank1.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_075000_rank3.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_075000_rank4.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_075000_rank5.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_075000_rank7.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_075000_rank6.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...eckpoints/model_075000.pt: 35%|███▍ | 818MB / 2.35GB  ...nts/optim_075000_rank2.pt: 87%|████████▋ | 2.16GB / 2.49GB  Processing Files (0 / 9) : 81%|████████▏ | 18.1GB / 22.2GB, 1.01GB/s New Data Upload : 75%|███████▍ | 4.11GB / 5.49GB, 211MB/s  ...nts/optim_075000_rank0.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_075000_rank1.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_075000_rank3.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_075000_rank4.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_075000_rank5.pt: 86%|████████▋ | 2.14GB / 2.49GB  ...nts/optim_075000_rank7.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_075000_rank6.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...eckpoints/model_075000.pt: 36%|███▌ | 839MB / 2.35GB  ...nts/optim_075000_rank2.pt: 87%|████████▋ | 2.17GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.2GB / 22.2GB, 1.01GB/s New Data Upload : 75%|███████▍ | 4.14GB / 5.56GB, 212MB/s  ...nts/optim_075000_rank0.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_075000_rank1.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_075000_rank3.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_075000_rank4.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_075000_rank5.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_075000_rank7.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_075000_rank6.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...eckpoints/model_075000.pt: 37%|███▋ | 858MB / 2.35GB  ...nts/optim_075000_rank2.pt: 88%|████████▊ | 2.18GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.3GB / 22.2GB, 1.00GB/s New Data Upload : 74%|███████▍ | 4.18GB / 5.62GB, 211MB/s  ...nts/optim_075000_rank0.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_075000_rank1.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_075000_rank3.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_075000_rank4.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_075000_rank5.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_075000_rank7.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_075000_rank6.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...eckpoints/model_075000.pt: 37%|███▋ | 880MB / 2.35GB  ...nts/optim_075000_rank2.pt: 88%|████████▊ | 2.19GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.4GB / 22.2GB, 995MB/s New Data Upload : 74%|███████▍ | 4.22GB / 5.69GB, 212MB/s  ...nts/optim_075000_rank0.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_075000_rank1.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_075000_rank3.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_075000_rank4.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_075000_rank5.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_075000_rank7.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_075000_rank6.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...eckpoints/model_075000.pt: 39%|███▊ | 909MB / 2.35GB  ...nts/optim_075000_rank2.pt: 89%|████████▊ | 2.20GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.5GB / 22.2GB, 990MB/s New Data Upload : 75%|███████▍ | 4.27GB / 5.69GB, 213MB/s  ...nts/optim_075000_rank0.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_075000_rank1.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_075000_rank3.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_075000_rank4.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_075000_rank5.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_075000_rank7.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_075000_rank6.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...eckpoints/model_075000.pt: 40%|███▉ | 932MB / 2.35GB  ...nts/optim_075000_rank2.pt: 89%|████████▉ | 2.22GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▍ | 18.6GB / 22.2GB, 983MB/s New Data Upload : 75%|███████▍ | 4.30GB / 5.76GB, 213MB/s  ...nts/optim_075000_rank0.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_075000_rank1.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_075000_rank3.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_075000_rank4.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_075000_rank5.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_075000_rank7.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_075000_rank6.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...eckpoints/model_075000.pt: 41%|████ | 964MB / 2.35GB  ...nts/optim_075000_rank2.pt: 90%|████████▉ | 2.22GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▍ | 18.8GB / 22.2GB, 970MB/s New Data Upload : 75%|███████▍ | 4.35GB / 5.83GB, 212MB/s  ...nts/optim_075000_rank0.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_075000_rank1.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_075000_rank3.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_075000_rank4.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_075000_rank5.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_075000_rank7.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_075000_rank6.pt: 90%|█████████ | 2.24GB / 2.49GB  ...eckpoints/model_075000.pt: 42%|████▏ | 992MB / 2.35GB  ...nts/optim_075000_rank2.pt: 90%|████████▉ | 2.24GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▍ | 18.9GB / 22.2GB, 960MB/s New Data Upload : 75%|███████▌ | 4.39GB / 5.83GB, 212MB/s  ...nts/optim_075000_rank0.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_075000_rank1.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_075000_rank3.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_075000_rank4.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_075000_rank5.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_075000_rank7.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_075000_rank6.pt: 91%|█████████ | 2.25GB / 2.49GB  ...eckpoints/model_075000.pt: 43%|████▎ | 1.02GB / 2.35GB  ...nts/optim_075000_rank2.pt: 90%|█████████ | 2.25GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▌ | 19.0GB / 22.2GB, 944MB/s New Data Upload : 75%|███████▌ | 4.43GB / 5.89GB, 211MB/s  ...nts/optim_075000_rank0.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_075000_rank1.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_075000_rank3.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_075000_rank4.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_075000_rank5.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_075000_rank7.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_075000_rank6.pt: 91%|█████████ | 2.26GB / 2.49GB  ...eckpoints/model_075000.pt: 45%|████▍ | 1.06GB / 2.35GB  ...nts/optim_075000_rank2.pt: 91%|█████████ | 2.26GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▌ | 19.1GB / 22.2GB, 932MB/s New Data Upload : 76%|███████▌ | 4.48GB / 5.89GB, 210MB/s  ...nts/optim_075000_rank0.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_075000_rank1.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_075000_rank3.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_075000_rank4.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_075000_rank5.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_075000_rank7.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_075000_rank6.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...eckpoints/model_075000.pt: 46%|████▌ | 1.08GB / 2.35GB  ...nts/optim_075000_rank2.pt: 91%|█████████ | 2.26GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▌ | 19.2GB / 22.2GB, 915MB/s New Data Upload : 76%|███████▌ | 4.51GB / 5.96GB, 208MB/s  ...nts/optim_075000_rank0.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_075000_rank1.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_075000_rank3.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_075000_rank4.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_075000_rank5.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_075000_rank7.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_075000_rank6.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...eckpoints/model_075000.pt: 48%|████▊ | 1.12GB / 2.35GB  ...nts/optim_075000_rank2.pt: 92%|█████████▏| 2.27GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.3GB / 22.2GB, 908MB/s New Data Upload : 76%|███████▌ | 4.57GB / 6.03GB, 209MB/s  ...nts/optim_075000_rank0.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_075000_rank1.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_075000_rank3.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_075000_rank4.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_075000_rank5.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_075000_rank7.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_075000_rank6.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...eckpoints/model_075000.pt: 49%|████▉ | 1.15GB / 2.35GB  ...nts/optim_075000_rank2.pt: 92%|█████████▏| 2.28GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.4GB / 22.2GB, 898MB/s New Data Upload : 77%|███████▋ | 4.61GB / 6.03GB, 208MB/s  ...nts/optim_075000_rank0.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_075000_rank1.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_075000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_075000_rank4.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_075000_rank5.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_075000_rank7.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_075000_rank6.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...eckpoints/model_075000.pt: 50%|█████ | 1.18GB / 2.35GB  ...nts/optim_075000_rank2.pt: 92%|█████████▏| 2.29GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.5GB / 22.2GB, 886MB/s New Data Upload : 76%|███████▌ | 4.65GB / 6.11GB, 207MB/s  ...nts/optim_075000_rank0.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_075000_rank1.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_075000_rank3.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_075000_rank4.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_075000_rank5.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_075000_rank7.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_075000_rank6.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...eckpoints/model_075000.pt: 52%|█████▏ | 1.22GB / 2.35GB  ...nts/optim_075000_rank2.pt: 93%|█████████▎| 2.30GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.6GB / 22.2GB, 880MB/s New Data Upload : 77%|███████▋ | 4.71GB / 6.11GB, 209MB/s  ...nts/optim_075000_rank0.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_075000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_075000_rank3.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_075000_rank4.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_075000_rank5.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_075000_rank7.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_075000_rank6.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...eckpoints/model_075000.pt: 53%|█████▎ | 1.25GB / 2.35GB  ...nts/optim_075000_rank2.pt: 93%|█████████▎| 2.31GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▊ | 19.7GB / 22.2GB, 869MB/s New Data Upload : 78%|███████▊ | 4.75GB / 6.11GB, 208MB/s  ...nts/optim_075000_rank0.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_075000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_075000_rank3.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_075000_rank4.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_075000_rank5.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_075000_rank7.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_075000_rank6.pt: 94%|█████████▎| 2.32GB / 2.49GB  ...eckpoints/model_075000.pt: 54%|█████▍ | 1.28GB / 2.35GB  ...nts/optim_075000_rank2.pt: 93%|█████████▎| 2.32GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▉ | 19.8GB / 22.2GB, 859MB/s New Data Upload : 78%|███████▊ | 4.79GB / 6.11GB, 207MB/s  ...nts/optim_075000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_075000_rank1.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_075000_rank3.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_075000_rank4.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_075000_rank5.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_075000_rank7.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_075000_rank6.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...eckpoints/model_075000.pt: 56%|█████▌ | 1.31GB / 2.35GB  ...nts/optim_075000_rank2.pt: 94%|█████████▎| 2.33GB / 2.49GB  Processing Files (0 / 9) : 90%|████████▉ | 19.9GB / 22.2GB, 851MB/s New Data Upload : 79%|███████▉ | 4.83GB / 6.11GB, 207MB/s  ...nts/optim_075000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_075000_rank1.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_075000_rank3.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_075000_rank4.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_075000_rank5.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_075000_rank7.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_075000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...eckpoints/model_075000.pt: 57%|█████▋ | 1.34GB / 2.35GB  ...nts/optim_075000_rank2.pt: 94%|█████████▍| 2.33GB / 2.49GB  Processing Files (0 / 9) : 90%|████████▉ | 20.0GB / 22.2GB, 845MB/s New Data Upload : 80%|███████▉ | 4.87GB / 6.11GB, 207MB/s  ...nts/optim_075000_rank0.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_075000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_075000_rank3.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_075000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_075000_rank5.pt: 94%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_075000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_075000_rank6.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...eckpoints/model_075000.pt: 58%|█████▊ | 1.36GB / 2.35GB  ...nts/optim_075000_rank2.pt: 94%|█████████▍| 2.34GB / 2.49GB  Processing Files (0 / 9) : 90%|█████████ | 20.1GB / 22.2GB, 834MB/s New Data Upload : 80%|████████ | 4.91GB / 6.11GB, 206MB/s  ...nts/optim_075000_rank0.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_075000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_075000_rank3.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_075000_rank4.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_075000_rank5.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_075000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_075000_rank6.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...eckpoints/model_075000.pt: 59%|█████▉ | 1.39GB / 2.35GB  ...nts/optim_075000_rank2.pt: 94%|█████████▍| 2.35GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.2GB / 22.2GB, 823MB/s New Data Upload : 81%|████████ | 4.94GB / 6.11GB, 205MB/s  ...nts/optim_075000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_075000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_075000_rank3.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_075000_rank4.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_075000_rank5.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_075000_rank7.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_075000_rank6.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...eckpoints/model_075000.pt: 60%|██████ | 1.41GB / 2.35GB  ...nts/optim_075000_rank2.pt: 95%|█████████▍| 2.36GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.3GB / 22.2GB, 816MB/s New Data Upload : 81%|████████▏ | 4.98GB / 6.11GB, 205MB/s  ...nts/optim_075000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_075000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_075000_rank3.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_075000_rank4.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_075000_rank5.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_075000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_075000_rank6.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...eckpoints/model_075000.pt: 61%|██████ | 1.43GB / 2.35GB  ...nts/optim_075000_rank2.pt: 95%|█████████▍| 2.36GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████▏| 20.3GB / 22.2GB, 803MB/s New Data Upload : 82%|████████▏ | 5.01GB / 6.11GB, 203MB/s  ...nts/optim_075000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_075000_rank1.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_075000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_075000_rank4.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_075000_rank5.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_075000_rank7.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_075000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...eckpoints/model_075000.pt: 62%|██████▏ | 1.46GB / 2.35GB  ...nts/optim_075000_rank2.pt: 95%|█████████▌| 2.37GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.4GB / 22.2GB, 801MB/s New Data Upload : 83%|████████▎ | 5.05GB / 6.11GB, 204MB/s  ...nts/optim_075000_rank0.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_075000_rank1.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_075000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_075000_rank5.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_075000_rank7.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_075000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...eckpoints/model_075000.pt: 63%|██████▎ | 1.48GB / 2.35GB  ...nts/optim_075000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.5GB / 22.2GB, 791MB/s New Data Upload : 83%|████████▎ | 5.08GB / 6.11GB, 202MB/s  ...nts/optim_075000_rank0.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_075000_rank1.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_075000_rank4.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_075000_rank5.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_075000_rank7.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_075000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...eckpoints/model_075000.pt: 64%|██████▍ | 1.51GB / 2.35GB  ...nts/optim_075000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.6GB / 22.2GB, 775MB/s New Data Upload : 84%|████████▎ | 5.12GB / 6.11GB, 202MB/s  ...nts/optim_075000_rank0.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank3.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_075000_rank4.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_075000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_075000_rank7.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_075000_rank6.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...eckpoints/model_075000.pt: 65%|██████▌ | 1.53GB / 2.35GB  ...nts/optim_075000_rank2.pt: 96%|█████████▌| 2.39GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.6GB / 22.2GB, 751MB/s New Data Upload : 84%|████████▍ | 5.15GB / 6.11GB, 202MB/s  ...nts/optim_075000_rank0.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank3.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_075000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_075000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_075000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...eckpoints/model_075000.pt: 66%|██████▋ | 1.56GB / 2.35GB  ...nts/optim_075000_rank2.pt: 96%|█████████▋| 2.40GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.7GB / 22.2GB, 733MB/s New Data Upload : 85%|████████▍ | 5.19GB / 6.11GB, 202MB/s  ...nts/optim_075000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_075000_rank3.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank5.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_075000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_075000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...eckpoints/model_075000.pt: 67%|██████▋ | 1.58GB / 2.35GB  ...nts/optim_075000_rank2.pt: 97%|█████████▋| 2.40GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▎| 20.8GB / 22.2GB, 708MB/s New Data Upload : 85%|████████▌ | 5.22GB / 6.11GB, 202MB/s  ...nts/optim_075000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_075000_rank1.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_075000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank5.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank7.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_075000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...eckpoints/model_075000.pt: 68%|██████▊ | 1.60GB / 2.35GB  ...nts/optim_075000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 20.9GB / 22.2GB, 682MB/s New Data Upload : 86%|████████▌ | 5.25GB / 6.11GB, 200MB/s  ...nts/optim_075000_rank0.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_075000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_075000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...eckpoints/model_075000.pt: 69%|██████▉ | 1.62GB / 2.35GB  ...nts/optim_075000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 21.0GB / 22.2GB, 660MB/s New Data Upload : 86%|████████▋ | 5.28GB / 6.11GB, 199MB/s  ...nts/optim_075000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_075000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_075000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...eckpoints/model_075000.pt: 70%|███████ | 1.64GB / 2.35GB  ...nts/optim_075000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▍| 21.0GB / 22.2GB, 649MB/s New Data Upload : 87%|████████▋ | 5.31GB / 6.11GB, 198MB/s  ...nts/optim_075000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank5.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_075000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...eckpoints/model_075000.pt: 71%|███████ | 1.66GB / 2.35GB  ...nts/optim_075000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▍| 21.1GB / 22.2GB, 628MB/s New Data Upload : 87%|████████▋ | 5.34GB / 6.11GB, 197MB/s  ...nts/optim_075000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_075000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...eckpoints/model_075000.pt: 72%|███████▏ | 1.69GB / 2.35GB  ...nts/optim_075000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.2GB / 22.2GB, 606MB/s New Data Upload : 88%|████████▊ | 5.37GB / 6.11GB, 196MB/s  ...nts/optim_075000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank1.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank7.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_075000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...eckpoints/model_075000.pt: 73%|███████▎ | 1.71GB / 2.35GB  ...nts/optim_075000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.2GB / 22.2GB, 587MB/s New Data Upload : 88%|████████▊ | 5.40GB / 6.11GB, 194MB/s  ...nts/optim_075000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank1.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...nts/optim_075000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_075000.pt: 74%|███████▎ | 1.73GB / 2.35GB  ...nts/optim_075000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 568MB/s New Data Upload : 89%|████████▉ | 5.43GB / 6.11GB, 193MB/s  ...nts/optim_075000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank4.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...nts/optim_075000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_075000.pt: 75%|███████▍ | 1.75GB / 2.35GB  ...nts/optim_075000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.4GB / 22.2GB, 558MB/s New Data Upload : 89%|████████▉ | 5.46GB / 6.11GB, 192MB/s  ...nts/optim_075000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_075000.pt: 76%|███████▌ | 1.78GB / 2.35GB  ...nts/optim_075000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 545MB/s New Data Upload : 90%|████████▉ | 5.50GB / 6.11GB, 192MB/s  ...nts/optim_075000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_075000.pt: 76%|███████▋ | 1.79GB / 2.35GB  ...nts/optim_075000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 525MB/s New Data Upload : 90%|█████████ | 5.52GB / 6.11GB, 189MB/s  ...nts/optim_075000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_075000.pt: 77%|███████▋ | 1.82GB / 2.35GB  ...nts/optim_075000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 498MB/s New Data Upload : 91%|█████████ | 5.55GB / 6.11GB, 188MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_075000.pt: 78%|███████▊ | 1.84GB / 2.35GB  ...nts/optim_075000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 487MB/s New Data Upload : 91%|█████████ | 5.58GB / 6.11GB, 186MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_075000.pt: 79%|███████▉ | 1.85GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.7GB / 22.2GB, 471MB/s New Data Upload : 92%|█████████▏| 5.60GB / 6.11GB, 185MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_075000.pt: 80%|███████▉ | 1.87GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 461MB/s New Data Upload : 92%|█████████▏| 5.62GB / 6.11GB, 183MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_075000.pt: 81%|████████ | 1.89GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 449MB/s New Data Upload : 92%|█████████▏| 5.64GB / 6.11GB, 181MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_075000.pt: 81%|████████ | 1.91GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 437MB/s New Data Upload : 93%|█████████▎| 5.66GB / 6.11GB, 178MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_075000.pt: 82%|████████▏ | 1.92GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 429MB/s New Data Upload : 93%|█████████▎| 5.68GB / 6.11GB, 176MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_075000.pt: 83%|████████▎ | 1.94GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 415MB/s New Data Upload : 93%|█████████▎| 5.70GB / 6.11GB, 174MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_075000.pt: 83%|████████▎ | 1.96GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 409MB/s New Data Upload : 94%|█████████▎| 5.72GB / 6.11GB, 172MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_075000.pt: 84%|████████▍ | 1.98GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 399MB/s New Data Upload : 94%|█████████▍| 5.74GB / 6.11GB, 170MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_075000.pt: 85%|████████▍ | 1.99GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 391MB/s New Data Upload : 94%|█████████▍| 5.75GB / 6.11GB, 168MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_075000.pt: 86%|████████▌ | 2.01GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 380MB/s New Data Upload : 94%|█████████▍| 5.77GB / 6.11GB, 166MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_075000.pt: 86%|████████▋ | 2.03GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 373MB/s New Data Upload : 95%|█████████▍| 5.79GB / 6.11GB, 165MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_075000.pt: 87%|████████▋ | 2.04GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 366MB/s New Data Upload : 95%|█████████▍| 5.81GB / 6.11GB, 163MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_075000.pt: 88%|████████▊ | 2.06GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 356MB/s New Data Upload : 95%|█████████▌| 5.82GB / 6.11GB, 161MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_075000.pt: 88%|████████▊ | 2.07GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 347MB/s New Data Upload : 95%|█████████▌| 5.83GB / 6.11GB, 158MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_075000.pt: 89%|████████▉ | 2.09GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 336MB/s New Data Upload : 96%|█████████▌| 5.86GB / 6.11GB, 156MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 90%|████████▉ | 2.11GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 328MB/s New Data Upload : 96%|█████████▌| 5.87GB / 6.11GB, 154MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 91%|█████████ | 2.13GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 319MB/s New Data Upload : 96%|█████████▋| 5.89GB / 6.11GB, 152MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 91%|█████████ | 2.14GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 309MB/s New Data Upload : 97%|█████████▋| 5.91GB / 6.11GB, 148MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 92%|█████████▏| 2.16GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 301MB/s New Data Upload : 97%|█████████▋| 5.92GB / 6.11GB, 146MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 92%|█████████▏| 2.17GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 291MB/s New Data Upload : 97%|█████████▋| 5.93GB / 6.11GB, 142MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 93%|█████████▎| 2.18GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 284MB/s New Data Upload : 97%|█████████▋| 5.95GB / 6.11GB, 141MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 94%|█████████▎| 2.20GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 272MB/s New Data Upload : 98%|█████████▊| 5.96GB / 6.11GB, 136MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 94%|█████████▍| 2.21GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 264MB/s New Data Upload : 98%|█████████▊| 5.98GB / 6.11GB, 134MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 95%|█████████▍| 2.22GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 257MB/s New Data Upload : 98%|█████████▊| 5.99GB / 6.11GB, 131MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 95%|█████████▌| 2.24GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 244MB/s New Data Upload : 98%|█████████▊| 6.00GB / 6.11GB, 127MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 96%|█████████▌| 2.25GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 238MB/s New Data Upload : 98%|█████████▊| 6.01GB / 6.11GB, 124MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 96%|█████████▌| 2.26GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 228MB/s New Data Upload : 98%|█████████▊| 6.02GB / 6.11GB, 121MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 96%|█████████▌| 2.26GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 219MB/s New Data Upload : 99%|█████████▊| 6.03GB / 6.11GB, 117MB/s  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 96%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 212MB/s New Data Upload : 99%|█████████▊| 6.03GB / 6.11GB, 114MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 203MB/s New Data Upload : 99%|█████████▊| 6.03GB / 6.11GB, 110MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 195MB/s New Data Upload : 99%|█████████▉| 6.04GB / 6.11GB, 107MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 187MB/s New Data Upload : 99%|█████████▉| 6.04GB / 6.11GB, 104MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 182MB/s New Data Upload : 99%|█████████▉| 6.05GB / 6.11GB, 102MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 172MB/s New Data Upload : 99%|█████████▉| 6.05GB / 6.11GB, 98.0MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 97%|█████████▋| 2.29GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 165MB/s New Data Upload : 99%|█████████▉| 6.05GB / 6.11GB, 95.7MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 98%|█████████▊| 2.29GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 158MB/s New Data Upload : 99%|█████████▉| 6.06GB / 6.11GB, 92.1MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 150MB/s New Data Upload : 99%|█████████▉| 6.06GB / 6.11GB, 89.2MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 143MB/s New Data Upload : 99%|█████████▉| 6.06GB / 6.11GB, 85.8MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 137MB/s New Data Upload : 99%|█████████▉| 6.07GB / 6.11GB, 83.5MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 129MB/s New Data Upload : 99%|█████████▉| 6.07GB / 6.11GB, 80.4MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 122MB/s New Data Upload : 99%|█████████▉| 6.07GB / 6.11GB, 77.6MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 99%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 116MB/s New Data Upload : 99%|█████████▉| 6.08GB / 6.11GB, 75.2MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 108MB/s New Data Upload : 99%|█████████▉| 6.08GB / 6.11GB, 72.5MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 102MB/s New Data Upload : 100%|█████████▉| 6.09GB / 6.11GB, 69.9MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 97.9MB/s New Data Upload : 100%|█████████▉| 6.09GB / 6.11GB, 67.7MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 88.4MB/s New Data Upload : 100%|█████████▉| 6.09GB / 6.11GB, 64.3MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 73.9MB/s New Data Upload : 100%|█████████▉| 6.09GB / 6.11GB, 58.1MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 70.6MB/s New Data Upload : 100%|█████████▉| 6.10GB / 6.11GB, 56.7MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 64.9MB/s New Data Upload : 100%|█████████▉| 6.10GB / 6.11GB, 53.8MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 99%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 59.0MB/s New Data Upload : 100%|█████████▉| 6.10GB / 6.11GB, 51.5MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 55.7MB/s New Data Upload : 100%|█████████▉| 6.10GB / 6.11GB, 49.7MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 52.4MB/s New Data Upload : 100%|█████████▉| 6.10GB / 6.11GB, 47.6MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 48.1MB/s New Data Upload : 100%|█████████▉| 6.11GB / 6.11GB, 43.9MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 45.7MB/s New Data Upload : 100%|█████████▉| 6.11GB / 6.11GB, 42.1MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 43.8MB/s New Data Upload : 100%|█████████▉| 6.11GB / 6.11GB, 40.2MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 41.4MB/s New Data Upload : 100%|█████████▉| 6.11GB / 6.11GB, 38.5MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 39.3MB/s New Data Upload : 100%|█████████▉| 6.11GB / 6.11GB, 36.6MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 37.5MB/s New Data Upload : 100%|█████████▉| 6.11GB / 6.11GB, 35.5MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 35.6MB/s New Data Upload : 100%|█████████▉| 6.11GB / 6.11GB, 33.6MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (9 / 9) : 100%|██████████| 22.2GB / 22.2GB, 23.6MB/s New Data Upload : 100%|██████████| 6.11GB / 6.11GB, 23.5MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (9 / 9) : 100%|██████████| 22.2GB / 22.2GB, 1.14MB/s New Data Upload : 100%|██████████| 6.11GB / 6.11GB, 1.14MB/s ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB [20:59:42] Step 75000 uploaded [21:59:42] Uploading step 80000... Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...nts/optim_075000_rank2.pt: 5%|▌ | 128MB / 2.49GB  ...nts/optim_075000_rank5.pt: 6%|▌ | 152MB / 2.49GB  ...nts/optim_075000_rank6.pt: 6%|▌ | 144MB / 2.49GB  ...nts/optim_075000_rank4.pt: 5%|▌ | 128MB / 2.49GB  ...nts/optim_075000_rank1.pt: 5%|▌ | 136MB / 2.49GB  ...nts/optim_075000_rank7.pt: 5%|▌ | 128MB / 2.49GB  ...nts/optim_075000_rank2.pt: 5%|▌ | 128MB / 2.49GB  ...nts/optim_075000_rank5.pt: 6%|▌ | 152MB / 2.49GB  ...nts/optim_075000_rank6.pt: 6%|▌ | 144MB / 2.49GB  ...nts/optim_075000_rank4.pt: 5%|▌ | 128MB / 2.49GB  ...nts/optim_075000_rank1.pt: 5%|▌ | 136MB / 2.49GB  ...nts/optim_075000_rank7.pt: 5%|▌ | 128MB / 2.49GB  Processing Files (0 / 6) : 2%|▏ | 816MB / 44.5GB, ???B/s ...nts/optim_075000_rank2.pt: 10%|▉ | 248MB / 2.49GB  ...nts/optim_075000_rank5.pt: 11%|█ | 272MB / 2.49GB  ...nts/optim_075000_rank6.pt: 10%|█ | 256MB / 2.49GB  ...nts/optim_075000_rank4.pt: 10%|▉ | 248MB / 2.49GB  ...nts/optim_075000_rank1.pt: 10%|▉ | 240MB / 2.49GB  ...nts/optim_075000_rank7.pt: 10%|▉ | 240MB / 2.49GB  Processing Files (0 / 6) : 3%|▎ | 1.50GB / 44.5GB, 3.45GB/s ...nts/optim_075000_rank2.pt: 14%|█▍ | 360MB / 2.49GB  ...nts/optim_075000_rank5.pt: 15%|█▌ | 384MB / 2.49GB  ...nts/optim_075000_rank6.pt: 15%|█▍ | 368MB / 2.49GB  ...nts/optim_075000_rank4.pt: 15%|█▌ | 376MB / 2.49GB  ...nts/optim_075000_rank1.pt: 14%|█▍ | 360MB / 2.49GB  ...nts/optim_075000_rank7.pt: 14%|█▍ | 352MB / 2.49GB  Processing Files (0 / 6) : 5%|▍ | 2.20GB / 44.5GB, 3.47GB/s ...nts/optim_075000_rank2.pt: 20%|█▉ | 496MB / 2.49GB  ...nts/optim_075000_rank5.pt: 21%|██ | 512MB / 2.49GB  ...nts/optim_075000_rank6.pt: 20%|█▉ | 496MB / 2.49GB  ...nts/optim_075000_rank4.pt: 20%|█▉ | 496MB / 2.49GB  ...nts/optim_075000_rank1.pt: 19%|█▉ | 480MB / 2.49GB  ...nts/optim_075000_rank7.pt: 19%|█▉ | 480MB / 2.49GB  Processing Files (0 / 6) : 7%|▋ | 2.96GB / 44.5GB, 3.57GB/s ...nts/optim_075000_rank2.pt: 25%|██▌ | 624MB / 2.49GB  ...nts/optim_075000_rank5.pt: 25%|██▌ | 632MB / 2.49GB  ...nts/optim_075000_rank6.pt: 25%|██▍ | 616MB / 2.49GB  ...nts/optim_075000_rank4.pt: 25%|██▍ | 616MB / 2.49GB  ...nts/optim_075000_rank1.pt: 24%|██▍ | 600MB / 2.49GB  ...nts/optim_075000_rank7.pt: 25%|██▍ | 616MB / 2.49GB  Processing Files (0 / 6) : 8%|▊ | 3.70GB / 44.5GB, 3.61GB/s ...nts/optim_075000_rank2.pt: 31%|███ | 760MB / 2.49GB  ...nts/optim_075000_rank5.pt: 31%|███ | 760MB / 2.49GB  ...nts/optim_075000_rank6.pt: 30%|██▉ | 744MB / 2.49GB  ...nts/optim_075000_rank4.pt: 30%|██▉ | 744MB / 2.49GB  ...nts/optim_075000_rank1.pt: 29%|██▉ | 720MB / 2.49GB  ...nts/optim_075000_rank7.pt: 30%|███ | 752MB / 2.49GB  Processing Files (0 / 6) : 10%|█ | 4.48GB / 44.5GB, 3.67GB/s ...nts/optim_080000_rank0.pt: 0%| | 335kB / 2.49GB  ...nts/optim_080000_rank7.pt: 0%| | 1.17MB / 2.49GB  ...nts/optim_075000_rank2.pt: 36%|███▌ | 888MB / 2.49GB  ...nts/optim_075000_rank5.pt: 35%|███▌ | 880MB / 2.49GB  ...nts/optim_075000_rank6.pt: 35%|███▌ | 872MB / 2.49GB  ...nts/optim_075000_rank4.pt: 35%|███▍ | 864MB / 2.49GB  ...nts/optim_075000_rank1.pt: 34%|███▍ | 848MB / 2.49GB  ...nts/optim_075000_rank7.pt: 35%|███▌ | 880MB / 2.49GB  ...nts/optim_080000_rank0.pt: 0%| | 335kB / 2.49GB  ...nts/optim_080000_rank7.pt: 0%| | 1.17MB / 2.49GB  Processing Files (0 / 8) : 12%|█▏ | 5.23GB / 44.5GB, 3.68GB/s New Data Upload : 1%| | 1.17MB / 201MB, 977kB/s  ...nts/optim_075000_rank2.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_075000_rank5.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_075000_rank6.pt: 40%|███▉ | 992MB / 2.49GB  ...nts/optim_075000_rank4.pt: 40%|███▉ | 992MB / 2.49GB  ...nts/optim_075000_rank1.pt: 40%|███▉ | 984MB / 2.49GB  ...nts/optim_075000_rank7.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_080000_rank0.pt: 0%| | 837kB / 2.49GB  ...nts/optim_080000_rank7.pt: 0%| | 2.93MB / 2.49GB  Processing Files (0 / 8) : 14%|█▎ | 6.00GB / 44.5GB, 3.71GB/s New Data Upload : 1%|▏ | 2.93MB / 201MB, 2.09MB/s  ...nts/optim_075000_rank2.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_075000_rank5.pt: 45%|████▌ | 1.12GB / 2.49GB  ...nts/optim_075000_rank6.pt: 45%|████▌ | 1.13GB / 2.49GB  ...nts/optim_075000_rank4.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_075000_rank1.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_075000_rank7.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_080000_rank0.pt: 0%| | 1.84MB / 2.49GB  ...nts/optim_080000_rank7.pt: 0%| | 6.45MB / 2.49GB  Processing Files (0 / 8) : 15%|█▌ | 6.75GB / 44.5GB, 3.71GB/s New Data Upload : 3%|▎ | 6.45MB / 201MB, 4.03MB/s  ...nts/optim_075000_rank2.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_075000_rank5.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_075000_rank6.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_075000_rank4.pt: 50%|████▉ | 1.23GB / 2.49GB  ...nts/optim_075000_rank1.pt: 50%|████▉ | 1.23GB / 2.49GB  ...nts/optim_075000_rank7.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_080000_rank0.pt: 0%| | 3.51MB / 2.49GB  ...nts/optim_080000_rank7.pt: 0%| | 12.3MB / 2.49GB  Processing Files (0 / 8) : 17%|█▋ | 7.48GB / 44.5GB, 3.70GB/s New Data Upload : 6%|▌ | 12.3MB / 201MB, 6.84MB/s  ...nts/optim_075000_rank2.pt: 56%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_075000_rank5.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_075000_rank6.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_075000_rank4.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_075000_rank1.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_075000_rank7.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_080000_rank0.pt: 0%| | 6.02MB / 2.49GB  ...nts/optim_080000_rank7.pt: 1%| | 21.1MB / 2.49GB  Processing Files (0 / 8) : 19%|█▊ | 8.25GB / 44.5GB, 3.72GB/s New Data Upload : 10%|█ | 21.1MB / 201MB, 10.6MB/s  ...nts/optim_075000_rank2.pt: 61%|██████ | 1.50GB / 2.49GB  ...nts/optim_075000_rank5.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_075000_rank6.pt: 61%|██████ | 1.50GB / 2.49GB  ...nts/optim_075000_rank4.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_075000_rank7.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_080000_rank0.pt: 0%| | 7.70MB / 2.49GB  ...nts/optim_080000_rank7.pt: 1%| | 27.0MB / 2.49GB  Processing Files (0 / 8) : 20%|██ | 9.00GB / 44.5GB, 3.72GB/s New Data Upload : 13%|█▎ | 27.0MB / 201MB, 12.3MB/s  ...nts/optim_075000_rank2.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_075000_rank5.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_075000_rank6.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_075000_rank4.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_075000_rank1.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_075000_rank7.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_080000_rank0.pt: 0%| | 7.70MB / 2.49GB  ...nts/optim_080000_rank7.pt: 1%| | 27.0MB / 2.49GB  Processing Files (0 / 8) : 22%|██▏ | 9.71GB / 44.5GB, 3.70GB/s ...nts/optim_075000_rank2.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_075000_rank5.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_075000_rank6.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_075000_rank4.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_075000_rank1.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_075000_rank7.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_080000_rank0.pt: 0%| | 9.04MB / 2.49GB  ...nts/optim_080000_rank7.pt: 1%|▏ | 31.7MB / 2.49GB  Processing Files (0 / 8) : 24%|██▎ | 10.5GB / 44.5GB, 3.71GB/s New Data Upload : 16%|█▌ | 31.7MB / 201MB, 12.2MB/s  ...nts/optim_075000_rank2.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_075000_rank5.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_075000_rank6.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_075000_rank4.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_075000_rank1.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_075000_rank7.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_080000_rank0.pt: 0%| | 11.2MB / 2.49GB  ...nts/optim_080000_rank7.pt: 2%|▏ | 39.3MB / 2.49GB  Processing Files (0 / 8) : 25%|██▌ | 11.2GB / 44.5GB, 3.71GB/s New Data Upload : 20%|█▉ | 39.3MB / 201MB, 14.0MB/s  ...nts/optim_075000_rank2.pt: 79%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_075000_rank5.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_075000_rank6.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_075000_rank4.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_075000_rank1.pt: 79%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_075000_rank7.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 12.7MB / 2.49GB  ...nts/optim_080000_rank7.pt: 2%|▏ | 44.6MB / 2.49GB  Processing Files (0 / 8) : 27%|██▋ | 12.0GB / 44.5GB, 3.71GB/s New Data Upload : 22%|██▏ | 44.6MB / 201MB, 14.9MB/s  ...nts/optim_075000_rank2.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_075000_rank5.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_075000_rank6.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_075000_rank4.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_075000_rank1.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_075000_rank7.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 14.1MB / 2.49GB  ...nts/optim_080000_rank7.pt: 2%|▏ | 49.2MB / 2.49GB  Processing Files (0 / 8) : 29%|██▊ | 12.7GB / 44.5GB, 3.71GB/s New Data Upload : 24%|██▍ | 49.2MB / 201MB, 15.4MB/s  ...nts/optim_075000_rank2.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_075000_rank5.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_075000_rank6.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_075000_rank4.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_075000_rank1.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_075000_rank7.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 14.1MB / 2.49GB  ...nts/optim_080000_rank7.pt: 2%|▏ | 49.2MB / 2.49GB  Processing Files (0 / 8) : 30%|███ | 13.4GB / 44.5GB, 3.71GB/s ...nts/optim_075000_rank2.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_075000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_075000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_075000_rank4.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_075000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_075000_rank7.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 15.4MB / 2.49GB  ...nts/optim_080000_rank7.pt: 2%|▏ | 53.9MB / 2.49GB  Processing Files (0 / 8) : 32%|███▏ | 14.2GB / 44.5GB, 3.71GB/s New Data Upload : 27%|██▋ | 53.9MB / 201MB, 15.0MB/s  ...nts/optim_075000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 16.4MB / 2.49GB  ...nts/optim_080000_rank7.pt: 2%|▏ | 57.5MB / 2.49GB  Processing Files (1 / 8) : 33%|███▎ | 14.9GB / 44.5GB, 3.70GB/s New Data Upload : 29%|██▊ | 57.5MB / 201MB, 15.1MB/s  ...eckpoints/model_075000.pt: 5%|▍ | 112MB / 2.35GB  ...nts/optim_075000_rank3.pt: 4%|▍ | 104MB / 2.49GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 17.4MB / 2.49GB  ...nts/optim_080000_rank7.pt: 2%|▏ | 61.0MB / 2.49GB  ...eckpoints/model_075000.pt: 5%|▍ | 112MB / 2.35GB  ...nts/optim_075000_rank3.pt: 4%|▍ | 104MB / 2.49GB  Processing Files (6 / 10) : 34%|███▍ | 15.2GB / 44.5GB, 3.60GB/s New Data Upload : 30%|███ | 61.0MB / 201MB, 15.2MB/s  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 18.4MB / 2.49GB  ...nts/optim_080000_rank7.pt: 3%|▎ | 64.5MB / 2.49GB  ...eckpoints/model_075000.pt: 10%|█ | 240MB / 2.35GB  [+ 4 files] : 2%|▏ | 290MB / 12.4GB  Processing Files (6 / 14) : 35%|███▍ | 15.5GB / 44.5GB, 3.50GB/s New Data Upload : 32%|███▏ | 64.5MB / 201MB, 15.4MB/s  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 18.9MB / 2.49GB  ...nts/optim_080000_rank7.pt: 3%|▎ | 66.3MB / 2.49GB  ...eckpoints/model_075000.pt: 16%|█▌ | 368MB / 2.35GB  ...nts/optim_075000_rank3.pt: 14%|█▍ | 352MB / 2.49GB  ...nts/optim_080000_rank3.pt: 1%| | 18.9MB / 2.49GB  ...nts/optim_080000_rank4.pt: 1%| | 18.9MB / 2.49GB  ...nts/optim_080000_rank5.pt: 1%| | 18.9MB / 2.49GB  ...nts/optim_080000_rank6.pt: 1%| | 18.9MB / 2.49GB  Processing Files (6 / 14) : 36%|███▌ | 15.8GB / 44.5GB, 3.40GB/s New Data Upload : 33%|███▎ | 66.3MB / 201MB, 15.1MB/s  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank7.pt: 3%|▎ | 66.8MB / 2.49GB  ...eckpoints/model_075000.pt: 21%|██ | 488MB / 2.35GB  ...nts/optim_075000_rank3.pt: 19%|█▊ | 464MB / 2.49GB  ...nts/optim_080000_rank3.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank4.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank5.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank6.pt: 1%| | 19.1MB / 2.49GB  Processing Files (6 / 14) : 36%|███▌ | 16.0GB / 44.5GB, 3.31GB/s New Data Upload : 25%|██▍ | 66.8MB / 268MB, 14.5MB/s  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank7.pt: 3%|▎ | 66.8MB / 2.49GB  ...eckpoints/model_075000.pt: 26%|██▌ | 616MB / 2.35GB  ...nts/optim_075000_rank3.pt: 24%|██▍ | 592MB / 2.49GB  ...nts/optim_080000_rank3.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank4.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank5.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank6.pt: 1%| | 19.1MB / 2.49GB  Processing Files (6 / 14) : 37%|███▋ | 16.3GB / 44.5GB, 3.22GB/s ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank7.pt: 3%|▎ | 66.8MB / 2.49GB  ...eckpoints/model_075000.pt: 32%|███▏ | 752MB / 2.35GB  ...nts/optim_075000_rank3.pt: 29%|██▉ | 728MB / 2.49GB  ...nts/optim_080000_rank3.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank4.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank5.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank6.pt: 1%| | 19.1MB / 2.49GB  Processing Files (6 / 14) : 37%|███▋ | 16.6GB / 44.5GB, 3.15GB/s ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank7.pt: 3%|▎ | 66.8MB / 2.49GB  ...eckpoints/model_075000.pt: 37%|███▋ | 880MB / 2.35GB  ...nts/optim_075000_rank3.pt: 34%|███▍ | 848MB / 2.49GB  ...nts/optim_080000_rank3.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank4.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank5.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank6.pt: 1%| | 19.1MB / 2.49GB  Processing Files (6 / 14) : 38%|███▊ | 16.8GB / 44.5GB, 3.07GB/s ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank7.pt: 3%|▎ | 66.8MB / 2.49GB  ...eckpoints/model_075000.pt: 43%|████▎ | 1.01GB / 2.35GB  ...nts/optim_075000_rank3.pt: 39%|███▉ | 968MB / 2.49GB  ...nts/optim_080000_rank3.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank4.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank5.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank6.pt: 1%| | 19.1MB / 2.49GB  Processing Files (6 / 14) : 38%|███▊ | 17.1GB / 44.5GB, 3.01GB/s ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank7.pt: 3%|▎ | 66.8MB / 2.49GB  ...eckpoints/model_075000.pt: 48%|████▊ | 1.13GB / 2.35GB  ...nts/optim_075000_rank3.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_080000_rank3.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank4.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank5.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_080000_rank6.pt: 1%| | 19.1MB / 2.49GB  Processing Files (6 / 14) : 39%|███▉ | 17.3GB / 44.5GB, 2.94GB/s ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 21.2MB / 2.49GB  ...nts/optim_080000_rank7.pt: 3%|▎ | 71.4MB / 2.49GB  ...eckpoints/model_075000.pt: 53%|█████▎ | 1.24GB / 2.35GB  ...nts/optim_075000_rank3.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_080000_rank3.pt: 1%| | 23.5MB / 2.49GB  ...nts/optim_080000_rank4.pt: 1%| | 23.5MB / 2.49GB  ...nts/optim_080000_rank5.pt: 1%| | 23.5MB / 2.49GB  ...nts/optim_080000_rank6.pt: 1%| | 23.5MB / 2.49GB  Processing Files (6 / 14) : 39%|███▉ | 17.6GB / 44.5GB, 2.89GB/s New Data Upload : 15%|█▌ | 71.4MB / 470MB, 12.3MB/s  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 21.8MB / 2.49GB  ...nts/optim_080000_rank7.pt: 3%|▎ | 71.5MB / 2.49GB  ...eckpoints/model_075000.pt: 59%|█████▊ | 1.38GB / 2.35GB  ...nts/optim_075000_rank3.pt: 54%|█████▎ | 1.34GB / 2.49GB  ...nts/optim_080000_rank3.pt: 1%| | 24.0MB / 2.49GB  ...nts/optim_080000_rank4.pt: 1%| | 24.0MB / 2.49GB  ...nts/optim_080000_rank5.pt: 1%| | 24.0MB / 2.49GB  ...nts/optim_080000_rank6.pt: 1%| | 24.0MB / 2.49GB  Processing Files (6 / 14) : 40%|████ | 17.8GB / 44.5GB, 2.83GB/s New Data Upload : 15%|█▌ | 72.0MB / 470MB, 12.0MB/s  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 21.8MB / 2.49GB  ...nts/optim_080000_rank7.pt: 3%|▎ | 71.5MB / 2.49GB  ...eckpoints/model_075000.pt: 64%|██████▍ | 1.50GB / 2.35GB  ...nts/optim_075000_rank3.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_080000_rank3.pt: 1%| | 24.0MB / 2.49GB  ...nts/optim_080000_rank4.pt: 1%| | 24.0MB / 2.49GB  ...nts/optim_080000_rank5.pt: 1%| | 24.0MB / 2.49GB  ...nts/optim_080000_rank6.pt: 1%| | 24.0MB / 2.49GB  Processing Files (6 / 14) : 41%|████ | 18.1GB / 44.5GB, 2.78GB/s ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%| | 28.9MB / 2.49GB  ...nts/optim_080000_rank7.pt: 3%|▎ | 76.1MB / 2.49GB  ...eckpoints/model_075000.pt: 69%|██████▉ | 1.63GB / 2.35GB  ...nts/optim_075000_rank3.pt: 63%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_080000_rank3.pt: 1%| | 30.2MB / 2.49GB  ...nts/optim_080000_rank4.pt: 1%| | 30.2MB / 2.49GB  ...nts/optim_080000_rank5.pt: 1%| | 30.2MB / 2.49GB  ...nts/optim_080000_rank6.pt: 1%| | 30.2MB / 2.49GB  Processing Files (6 / 14) : 41%|████▏ | 18.3GB / 44.5GB, 2.74GB/s New Data Upload : 15%|█▍ | 78.7MB / 537MB, 12.3MB/s  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 1%|▏ | 34.1MB / 2.49GB  ...nts/optim_080000_rank7.pt: 3%|▎ | 80.0MB / 2.49GB  ...eckpoints/model_075000.pt: 75%|███████▍ | 1.76GB / 2.35GB  ...nts/optim_075000_rank3.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_080000_rank3.pt: 1%|▏ | 35.8MB / 2.49GB  ...nts/optim_080000_rank4.pt: 1%|▏ | 35.8MB / 2.49GB  ...nts/optim_080000_rank5.pt: 1%|▏ | 35.8MB / 2.49GB  ...nts/optim_080000_rank6.pt: 1%|▏ | 35.8MB / 2.49GB  Processing Files (6 / 14) : 42%|████▏ | 18.7GB / 44.5GB, 2.70GB/s New Data Upload : 14%|█▍ | 84.8MB / 604MB, 12.8MB/s  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 2%|▏ | 40.7MB / 2.49GB  ...nts/optim_080000_rank7.pt: 3%|▎ | 84.9MB / 2.49GB  ...eckpoints/model_075000.pt: 81%|████████ | 1.90GB / 2.35GB  ...nts/optim_075000_rank3.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_080000_rank3.pt: 2%|▏ | 43.2MB / 2.49GB  ...nts/optim_080000_rank4.pt: 2%|▏ | 42.7MB / 2.49GB  ...nts/optim_080000_rank5.pt: 2%|▏ | 42.8MB / 2.49GB  ...nts/optim_080000_rank6.pt: 2%|▏ | 43.0MB / 2.49GB  Processing Files (6 / 14) : 43%|████▎ | 19.0GB / 44.5GB, 2.67GB/s New Data Upload : 14%|█▍ | 92.8MB / 671MB, 13.6MB/s  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 2%|▏ | 51.9MB / 2.49GB  ...nts/optim_080000_rank7.pt: 4%|▎ | 90.6MB / 2.49GB  ...eckpoints/model_075000.pt: 87%|████████▋ | 2.05GB / 2.35GB  ...nts/optim_075000_rank3.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_080000_rank3.pt: 2%|▏ | 55.5MB / 2.49GB  ...nts/optim_080000_rank4.pt: 2%|▏ | 53.5MB / 2.49GB  ...nts/optim_080000_rank5.pt: 2%|▏ | 53.9MB / 2.49GB  ...nts/optim_080000_rank6.pt: 2%|▏ | 54.6MB / 2.49GB  Processing Files (6 / 14) : 43%|████▎ | 19.3GB / 44.5GB, 2.64GB/s New Data Upload : 16%|█▌ | 106MB / 671MB, 15.2MB/s  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 2%|▏ | 61.0MB / 2.49GB  ...nts/optim_080000_rank7.pt: 4%|▍ | 94.9MB / 2.49GB  ...eckpoints/model_075000.pt: 93%|█████████▎| 2.18GB / 2.35GB  ...nts/optim_075000_rank3.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_080000_rank3.pt: 3%|▎ | 67.3MB / 2.49GB  ...nts/optim_080000_rank4.pt: 2%|▏ | 61.9MB / 2.49GB  ...nts/optim_080000_rank5.pt: 3%|▎ | 62.9MB / 2.49GB  ...nts/optim_080000_rank6.pt: 3%|▎ | 64.9MB / 2.49GB  Processing Files (6 / 14) : 44%|████▍ | 19.6GB / 44.5GB, 2.61GB/s New Data Upload : 16%|█▌ | 119MB / 738MB, 16.6MB/s  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 3%|▎ | 75.3MB / 2.49GB  ...nts/optim_080000_rank7.pt: 4%|▍ | 103MB / 2.49GB  ...eckpoints/model_075000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_075000_rank3.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_080000_rank3.pt: 3%|▎ | 85.8MB / 2.49GB  ...nts/optim_080000_rank4.pt: 3%|▎ | 77.1MB / 2.49GB  ...nts/optim_080000_rank5.pt: 3%|▎ | 77.5MB / 2.49GB  ...nts/optim_080000_rank6.pt: 3%|▎ | 81.2MB / 2.49GB  Processing Files (6 / 14) : 45%|████▍ | 20.0GB / 44.5GB, 2.59GB/s New Data Upload : 19%|█▉ | 141MB / 738MB, 19.0MB/s  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 4%|▎ | 88.2MB / 2.49GB  ...nts/optim_080000_rank7.pt: 5%|▍ | 113MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_080000_rank3.pt: 4%|▍ | 105MB / 2.49GB  ...nts/optim_080000_rank4.pt: 4%|▍ | 93.6MB / 2.49GB  ...nts/optim_080000_rank5.pt: 4%|▎ | 91.5MB / 2.49GB  ...nts/optim_080000_rank6.pt: 4%|▍ | 97.4MB / 2.49GB  Processing Files (7 / 14) : 46%|████▌ | 20.2GB / 44.5GB, 2.56GB/s New Data Upload : 23%|██▎ | 166MB / 738MB, 21.9MB/s  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 4%|▍ | 102MB / 2.49GB  ...nts/optim_080000_rank7.pt: 5%|▍ | 119MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 5%|▍ | 119MB / 2.49GB  ...nts/optim_080000_rank4.pt: 4%|▍ | 110MB / 2.49GB  ...nts/optim_080000_rank5.pt: 4%|▍ | 104MB / 2.49GB  [+ 1 files] : 3%|▎ | 171MB / 4.97GB  Processing Files (8 / 15) : 46%|████▌ | 20.5GB / 44.5GB, 2.52GB/s New Data Upload : 23%|██▎ | 186MB / 805MB, 23.9MB/s  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank0.pt: 5%|▍ | 119MB / 2.49GB  ...nts/optim_080000_rank7.pt: 5%|▌ | 129MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 6%|▌ | 139MB / 2.49GB  ...nts/optim_080000_rank4.pt: 5%|▌ | 127MB / 2.49GB  ...nts/optim_080000_rank5.pt: 5%|▍ | 120MB / 2.49GB  ...nts/optim_080000_rank6.pt: 5%|▌ | 128MB / 2.49GB  [+ 1 files] : 2%|▏ | 99.5MB / 4.97GB  Processing Files (8 / 16) : 46%|████▋ | 20.6GB / 44.5GB, 2.47GB/s New Data Upload : 27%|██▋ | 215MB / 805MB, 26.9MB/s  ...nts/optim_080000_rank0.pt: 6%|▌ | 137MB / 2.49GB  ...nts/optim_080000_rank7.pt: 6%|▌ | 141MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 6%|▋ | 161MB / 2.49GB  ...nts/optim_080000_rank4.pt: 6%|▌ | 148MB / 2.49GB  ...nts/optim_080000_rank5.pt: 6%|▌ | 140MB / 2.49GB  ...nts/optim_080000_rank6.pt: 6%|▌ | 148MB / 2.49GB  ...nts/optim_080000_rank1.pt: 5%|▌ | 131MB / 2.49GB  ...nts/optim_080000_rank2.pt: 3%|▎ | 75.7MB / 2.49GB  Processing Files (8 / 16) : 47%|████▋ | 20.8GB / 44.5GB, 2.44GB/s New Data Upload : 31%|███ | 248MB / 805MB, 30.2MB/s  ...nts/optim_080000_rank0.pt: 6%|▌ | 151MB / 2.49GB  ...nts/optim_080000_rank7.pt: 6%|▌ | 152MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 7%|▋ | 181MB / 2.49GB  ...nts/optim_080000_rank4.pt: 7%|▋ | 169MB / 2.49GB  ...nts/optim_080000_rank5.pt: 6%|▋ | 155MB / 2.49GB  ...nts/optim_080000_rank6.pt: 7%|▋ | 165MB / 2.49GB  ...nts/optim_080000_rank1.pt: 6%|▌ | 145MB / 2.49GB  ...nts/optim_080000_rank2.pt: 6%|▌ | 144MB / 2.49GB  Processing Files (8 / 16) : 47%|████▋ | 21.0GB / 44.5GB, 2.40GB/s New Data Upload : 32%|███▏ | 280MB / 872MB, 33.3MB/s  ...nts/optim_080000_rank0.pt: 7%|▋ | 167MB / 2.49GB  ...nts/optim_080000_rank7.pt: 7%|▋ | 166MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 8%|▊ | 197MB / 2.49GB  ...nts/optim_080000_rank4.pt: 8%|▊ | 187MB / 2.49GB  ...nts/optim_080000_rank5.pt: 7%|▋ | 174MB / 2.49GB  ...nts/optim_080000_rank6.pt: 7%|▋ | 178MB / 2.49GB  ...nts/optim_080000_rank1.pt: 7%|▋ | 163MB / 2.49GB  ...nts/optim_080000_rank2.pt: 7%|▋ | 163MB / 2.49GB  Processing Files (8 / 16) : 48%|████▊ | 21.1GB / 44.5GB, 2.36GB/s New Data Upload : 36%|███▌ | 314MB / 872MB, 36.5MB/s  ...nts/optim_080000_rank0.pt: 7%|▋ | 181MB / 2.49GB  ...nts/optim_080000_rank7.pt: 7%|▋ | 180MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 8%|▊ | 207MB / 2.49GB  ...nts/optim_080000_rank4.pt: 8%|▊ | 205MB / 2.49GB  ...nts/optim_080000_rank5.pt: 8%|▊ | 192MB / 2.49GB  ...nts/optim_080000_rank6.pt: 8%|▊ | 189MB / 2.49GB  ...nts/optim_080000_rank1.pt: 7%|▋ | 177MB / 2.49GB  ...nts/optim_080000_rank2.pt: 7%|▋ | 177MB / 2.49GB  Processing Files (8 / 16) : 48%|████▊ | 21.3GB / 44.5GB, 2.32GB/s New Data Upload : 37%|███▋ | 344MB / 939MB, 39.0MB/s  ...nts/optim_080000_rank0.pt: 8%|▊ | 196MB / 2.49GB  ...nts/optim_080000_rank7.pt: 8%|▊ | 195MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 9%|▊ | 217MB / 2.49GB  ...nts/optim_080000_rank4.pt: 9%|▉ | 222MB / 2.49GB  ...nts/optim_080000_rank5.pt: 8%|▊ | 205MB / 2.49GB  ...nts/optim_080000_rank6.pt: 8%|▊ | 201MB / 2.49GB  ...nts/optim_080000_rank1.pt: 8%|▊ | 191MB / 2.49GB  ...nts/optim_080000_rank2.pt: 8%|▊ | 191MB / 2.49GB  Processing Files (8 / 16) : 48%|████▊ | 21.4GB / 44.5GB, 2.28GB/s New Data Upload : 39%|███▉ | 370MB / 939MB, 41.2MB/s  ...nts/optim_080000_rank0.pt: 9%|▊ | 217MB / 2.49GB  ...nts/optim_080000_rank7.pt: 9%|▉ | 218MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 9%|▉ | 235MB / 2.49GB  ...nts/optim_080000_rank4.pt: 10%|▉ | 239MB / 2.49GB  ...nts/optim_080000_rank5.pt: 9%|▉ | 225MB / 2.49GB  ...nts/optim_080000_rank6.pt: 9%|▉ | 220MB / 2.49GB  ...nts/optim_080000_rank1.pt: 9%|▊ | 212MB / 2.49GB  ...nts/optim_080000_rank2.pt: 9%|▊ | 212MB / 2.49GB  Processing Files (8 / 16) : 48%|████▊ | 21.5GB / 44.5GB, 2.25GB/s New Data Upload : 43%|████▎ | 403MB / 939MB, 43.8MB/s  ...nts/optim_080000_rank0.pt: 10%|▉ | 238MB / 2.49GB  ...nts/optim_080000_rank7.pt: 10%|▉ | 241MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 10%|█ | 254MB / 2.49GB  ...nts/optim_080000_rank4.pt: 10%|█ | 255MB / 2.49GB  ...nts/optim_080000_rank5.pt: 10%|▉ | 245MB / 2.49GB  ...nts/optim_080000_rank6.pt: 10%|▉ | 242MB / 2.49GB  ...nts/optim_080000_rank1.pt: 9%|▉ | 234MB / 2.49GB  ...nts/optim_080000_rank2.pt: 9%|▉ | 234MB / 2.49GB  Processing Files (8 / 16) : 49%|████▉ | 21.7GB / 44.5GB, 2.22GB/s New Data Upload : 43%|████▎ | 433MB / 1.01GB, 46.0MB/s  ...nts/optim_080000_rank0.pt: 10%|█ | 260MB / 2.49GB  ...nts/optim_080000_rank7.pt: 11%|█ | 268MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 11%|█ | 278MB / 2.49GB  ...nts/optim_080000_rank4.pt: 11%|█ | 273MB / 2.49GB  ...nts/optim_080000_rank5.pt: 11%|█ | 269MB / 2.49GB  ...nts/optim_080000_rank6.pt: 11%|█ | 264MB / 2.49GB  ...nts/optim_080000_rank1.pt: 10%|█ | 258MB / 2.49GB  ...nts/optim_080000_rank2.pt: 10%|█ | 258MB / 2.49GB  Processing Files (8 / 16) : 49%|████▉ | 21.9GB / 44.5GB, 2.19GB/s New Data Upload : 43%|████▎ | 463MB / 1.07GB, 48.2MB/s  ...nts/optim_080000_rank0.pt: 11%|█▏ | 281MB / 2.49GB  ...nts/optim_080000_rank7.pt: 12%|█▏ | 297MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 12%|█▏ | 303MB / 2.49GB  ...nts/optim_080000_rank4.pt: 12%|█▏ | 291MB / 2.49GB  ...nts/optim_080000_rank5.pt: 12%|█▏ | 292MB / 2.49GB  ...nts/optim_080000_rank6.pt: 12%|█▏ | 287MB / 2.49GB  ...nts/optim_080000_rank1.pt: 11%|█▏ | 282MB / 2.49GB  ...nts/optim_080000_rank2.pt: 11%|█▏ | 282MB / 2.49GB  Processing Files (8 / 16) : 50%|████▉ | 22.1GB / 44.5GB, 2.17GB/s New Data Upload : 46%|████▌ | 495MB / 1.07GB, 50.5MB/s  ...nts/optim_080000_rank0.pt: 12%|█▏ | 300MB / 2.49GB  ...nts/optim_080000_rank7.pt: 13%|█▎ | 321MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 13%|█▎ | 327MB / 2.49GB  ...nts/optim_080000_rank4.pt: 12%|█▏ | 309MB / 2.49GB  ...nts/optim_080000_rank5.pt: 12%|█▏ | 310MB / 2.49GB  ...nts/optim_080000_rank6.pt: 12%|█▏ | 307MB / 2.49GB  ...nts/optim_080000_rank1.pt: 12%|█▏ | 302MB / 2.49GB  ...nts/optim_080000_rank2.pt: 12%|█▏ | 302MB / 2.49GB  Processing Files (8 / 16) : 50%|████▉ | 22.2GB / 44.5GB, 2.14GB/s New Data Upload : 46%|████▌ | 522MB / 1.14GB, 52.3MB/s  ...nts/optim_080000_rank0.pt: 13%|█▎ | 324MB / 2.49GB  ...nts/optim_080000_rank7.pt: 14%|█▍ | 348MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 14%|█▍ | 352MB / 2.49GB  ...nts/optim_080000_rank4.pt: 13%|█▎ | 332MB / 2.49GB  ...nts/optim_080000_rank5.pt: 13%|█▎ | 334MB / 2.49GB  ...nts/optim_080000_rank6.pt: 13%|█▎ | 332MB / 2.49GB  ...nts/optim_080000_rank1.pt: 13%|█▎ | 327MB / 2.49GB  ...nts/optim_080000_rank2.pt: 13%|█▎ | 328MB / 2.49GB  Processing Files (8 / 16) : 50%|█████ | 22.4GB / 44.5GB, 2.12GB/s New Data Upload : 43%|████▎ | 552MB / 1.27GB, 54.1MB/s  ...nts/optim_080000_rank0.pt: 14%|█▍ | 350MB / 2.49GB  ...nts/optim_080000_rank7.pt: 15%|█▌ | 377MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 15%|█▌ | 381MB / 2.49GB  ...nts/optim_080000_rank4.pt: 14%|█▍ | 357MB / 2.49GB  ...nts/optim_080000_rank5.pt: 14%|█▍ | 359MB / 2.49GB  ...nts/optim_080000_rank6.pt: 15%|█▍ | 361MB / 2.49GB  ...nts/optim_080000_rank1.pt: 14%|█▍ | 356MB / 2.49GB  ...nts/optim_080000_rank2.pt: 14%|█▍ | 356MB / 2.49GB  Processing Files (8 / 16) : 51%|█████ | 22.6GB / 44.5GB, 2.07GB/s New Data Upload : 40%|███▉ | 584MB / 1.48GB, 57.3MB/s  ...nts/optim_080000_rank0.pt: 15%|█▍ | 373MB / 2.49GB  ...nts/optim_080000_rank7.pt: 16%|█▌ | 402MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 16%|█▌ | 402MB / 2.49GB  ...nts/optim_080000_rank4.pt: 15%|█▌ | 378MB / 2.49GB  ...nts/optim_080000_rank5.pt: 15%|█▌ | 380MB / 2.49GB  ...nts/optim_080000_rank6.pt: 15%|█▌ | 382MB / 2.49GB  ...nts/optim_080000_rank1.pt: 15%|█▌ | 377MB / 2.49GB  ...nts/optim_080000_rank2.pt: 15%|█▌ | 377MB / 2.49GB  Processing Files (8 / 16) : 51%|█████▏ | 22.8GB / 44.5GB, 2.02GB/s New Data Upload : 41%|████▏ | 611MB / 1.48GB, 59.9MB/s  ...nts/optim_080000_rank0.pt: 16%|█▌ | 388MB / 2.49GB  ...nts/optim_080000_rank7.pt: 17%|█▋ | 421MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 17%|█▋ | 421MB / 2.49GB  ...nts/optim_080000_rank4.pt: 16%|█▌ | 394MB / 2.49GB  ...nts/optim_080000_rank5.pt: 16%|█▌ | 396MB / 2.49GB  ...nts/optim_080000_rank6.pt: 16%|█▌ | 397MB / 2.49GB  ...nts/optim_080000_rank1.pt: 16%|█▌ | 393MB / 2.49GB  ...nts/optim_080000_rank2.pt: 16%|█▌ | 393MB / 2.49GB  Processing Files (8 / 16) : 52%|█████▏ | 22.9GB / 44.5GB, 1.96GB/s New Data Upload : 43%|████▎ | 632MB / 1.48GB, 62.0MB/s  ...nts/optim_080000_rank0.pt: 17%|█▋ | 411MB / 2.49GB  ...nts/optim_080000_rank7.pt: 18%|█▊ | 447MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 18%|█▊ | 444MB / 2.49GB  ...nts/optim_080000_rank4.pt: 17%|█▋ | 416MB / 2.49GB  ...nts/optim_080000_rank5.pt: 17%|█▋ | 418MB / 2.49GB  ...nts/optim_080000_rank6.pt: 17%|█▋ | 420MB / 2.49GB  ...nts/optim_080000_rank1.pt: 17%|█▋ | 416MB / 2.49GB  ...nts/optim_080000_rank2.pt: 17%|█▋ | 416MB / 2.49GB  Processing Files (8 / 16) : 52%|█████▏ | 23.1GB / 44.5GB, 1.91GB/s New Data Upload : 43%|████▎ | 661MB / 1.54GB, 64.9MB/s  ...nts/optim_080000_rank0.pt: 18%|█▊ | 438MB / 2.49GB  ...nts/optim_080000_rank7.pt: 19%|█▉ | 474MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 19%|█▉ | 470MB / 2.49GB  ...nts/optim_080000_rank4.pt: 18%|█▊ | 438MB / 2.49GB  ...nts/optim_080000_rank5.pt: 18%|█▊ | 440MB / 2.49GB  ...nts/optim_080000_rank6.pt: 18%|█▊ | 445MB / 2.49GB  ...nts/optim_080000_rank1.pt: 18%|█▊ | 443MB / 2.49GB  ...nts/optim_080000_rank2.pt: 18%|█▊ | 440MB / 2.49GB  Processing Files (8 / 16) : 52%|█████▏ | 23.3GB / 44.5GB, 1.85GB/s New Data Upload : 43%|████▎ | 695MB / 1.61GB, 68.1MB/s  ...nts/optim_080000_rank0.pt: 19%|█▊ | 466MB / 2.49GB  ...nts/optim_080000_rank7.pt: 20%|██ | 504MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 20%|██ | 498MB / 2.49GB  ...nts/optim_080000_rank4.pt: 19%|█▊ | 463MB / 2.49GB  ...nts/optim_080000_rank5.pt: 19%|█▊ | 465MB / 2.49GB  ...nts/optim_080000_rank6.pt: 19%|█▉ | 472MB / 2.49GB  ...nts/optim_080000_rank1.pt: 19%|█▉ | 469MB / 2.49GB  ...nts/optim_080000_rank2.pt: 19%|█▊ | 465MB / 2.49GB  Processing Files (8 / 16) : 53%|█████▎ | 23.6GB / 44.5GB, 1.80GB/s New Data Upload : 44%|████▎ | 731MB / 1.68GB, 71.5MB/s  ...nts/optim_080000_rank0.pt: 20%|█▉ | 494MB / 2.49GB  ...nts/optim_080000_rank7.pt: 22%|██▏ | 536MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 21%|██▏ | 529MB / 2.49GB  ...nts/optim_080000_rank4.pt: 20%|█▉ | 493MB / 2.49GB  ...nts/optim_080000_rank5.pt: 20%|█▉ | 494MB / 2.49GB  ...nts/optim_080000_rank6.pt: 20%|██ | 501MB / 2.49GB  ...nts/optim_080000_rank1.pt: 20%|██ | 499MB / 2.49GB  ...nts/optim_080000_rank2.pt: 20%|█▉ | 495MB / 2.49GB  Processing Files (8 / 16) : 53%|█████▎ | 23.8GB / 44.5GB, 1.74GB/s New Data Upload : 42%|████▏ | 770MB / 1.81GB, 75.2MB/s  ...nts/optim_080000_rank0.pt: 21%|██ | 512MB / 2.49GB  ...nts/optim_080000_rank7.pt: 22%|██▏ | 556MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 22%|██▏ | 552MB / 2.49GB  ...nts/optim_080000_rank4.pt: 21%|██ | 513MB / 2.49GB  ...nts/optim_080000_rank5.pt: 21%|██ | 513MB / 2.49GB  ...nts/optim_080000_rank6.pt: 21%|██ | 521MB / 2.49GB  ...nts/optim_080000_rank1.pt: 21%|██ | 519MB / 2.49GB  ...nts/optim_080000_rank2.pt: 21%|██ | 515MB / 2.49GB  Processing Files (8 / 16) : 54%|█████▍ | 23.9GB / 44.5GB, 1.69GB/s New Data Upload : 44%|████▍ | 800MB / 1.81GB, 77.8MB/s  ...nts/optim_080000_rank0.pt: 21%|██▏ | 534MB / 2.49GB  ...nts/optim_080000_rank7.pt: 23%|██▎ | 576MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 23%|██▎ | 569MB / 2.49GB  ...nts/optim_080000_rank4.pt: 21%|██▏ | 534MB / 2.49GB  ...nts/optim_080000_rank5.pt: 21%|██▏ | 531MB / 2.49GB  ...nts/optim_080000_rank6.pt: 22%|██▏ | 540MB / 2.49GB  ...nts/optim_080000_rank1.pt: 22%|██▏ | 539MB / 2.49GB  ...nts/optim_080000_rank2.pt: 21%|██▏ | 532MB / 2.49GB  Processing Files (8 / 16) : 54%|█████▍ | 24.1GB / 44.5GB, 1.63GB/s New Data Upload : 44%|████▍ | 827MB / 1.88GB, 79.9MB/s  ...nts/optim_080000_rank0.pt: 22%|██▏ | 550MB / 2.49GB  ...nts/optim_080000_rank7.pt: 24%|██▍ | 591MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 23%|██▎ | 580MB / 2.49GB  ...nts/optim_080000_rank4.pt: 22%|██▏ | 546MB / 2.49GB  ...nts/optim_080000_rank5.pt: 22%|██▏ | 545MB / 2.49GB  ...nts/optim_080000_rank6.pt: 22%|██▏ | 554MB / 2.49GB  ...nts/optim_080000_rank1.pt: 22%|██▏ | 553MB / 2.49GB  ...nts/optim_080000_rank2.pt: 22%|██▏ | 546MB / 2.49GB  Processing Files (8 / 16) : 54%|█████▍ | 24.2GB / 44.5GB, 1.56GB/s New Data Upload : 45%|████▍ | 844MB / 1.88GB, 80.7MB/s  ...nts/optim_080000_rank0.pt: 23%|██▎ | 567MB / 2.49GB  ...nts/optim_080000_rank7.pt: 24%|██▍ | 606MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 24%|██▍ | 592MB / 2.49GB  ...nts/optim_080000_rank4.pt: 23%|██▎ | 562MB / 2.49GB  ...nts/optim_080000_rank5.pt: 23%|██▎ | 560MB / 2.49GB  ...nts/optim_080000_rank6.pt: 23%|██▎ | 569MB / 2.49GB  ...nts/optim_080000_rank1.pt: 23%|██▎ | 568MB / 2.49GB  ...nts/optim_080000_rank2.pt: 23%|██▎ | 560MB / 2.49GB  Processing Files (8 / 16) : 55%|█████▍ | 24.3GB / 44.5GB, 1.50GB/s New Data Upload : 46%|████▌ | 867MB / 1.88GB, 82.3MB/s  ...nts/optim_080000_rank0.pt: 23%|██▎ | 579MB / 2.49GB  ...nts/optim_080000_rank7.pt: 25%|██▍ | 617MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 24%|██▍ | 600MB / 2.49GB  ...nts/optim_080000_rank4.pt: 23%|██▎ | 576MB / 2.49GB  ...nts/optim_080000_rank5.pt: 23%|██▎ | 572MB / 2.49GB  ...nts/optim_080000_rank6.pt: 23%|██▎ | 579MB / 2.49GB  ...nts/optim_080000_rank1.pt: 23%|██▎ | 577MB / 2.49GB  ...nts/optim_080000_rank2.pt: 23%|██▎ | 570MB / 2.49GB  Processing Files (8 / 16) : 55%|█████▍ | 24.4GB / 44.5GB, 1.44GB/s New Data Upload : 45%|████▌ | 884MB / 1.95GB, 84.1MB/s  ...nts/optim_080000_rank0.pt: 24%|██▍ | 602MB / 2.49GB  ...nts/optim_080000_rank7.pt: 26%|██▌ | 638MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 25%|██▌ | 622MB / 2.49GB  ...nts/optim_080000_rank4.pt: 24%|██▍ | 599MB / 2.49GB  ...nts/optim_080000_rank5.pt: 24%|██▍ | 598MB / 2.49GB  ...nts/optim_080000_rank6.pt: 24%|██▍ | 600MB / 2.49GB  ...nts/optim_080000_rank1.pt: 24%|██▍ | 600MB / 2.49GB  ...nts/optim_080000_rank2.pt: 24%|██▍ | 595MB / 2.49GB  Processing Files (8 / 16) : 55%|█████▌ | 24.6GB / 44.5GB, 1.39GB/s New Data Upload : 47%|████▋ | 916MB / 1.95GB, 86.7MB/s  ...nts/optim_080000_rank0.pt: 25%|██▌ | 627MB / 2.49GB  ...nts/optim_080000_rank7.pt: 27%|██▋ | 662MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 26%|██▌ | 642MB / 2.49GB  ...nts/optim_080000_rank4.pt: 25%|██▌ | 626MB / 2.49GB  ...nts/optim_080000_rank5.pt: 25%|██▌ | 626MB / 2.49GB  ...nts/optim_080000_rank6.pt: 25%|██▌ | 622MB / 2.49GB  ...nts/optim_080000_rank1.pt: 25%|██▌ | 623MB / 2.49GB  ...nts/optim_080000_rank2.pt: 25%|██▍ | 621MB / 2.49GB  Processing Files (8 / 16) : 56%|█████▌ | 24.8GB / 44.5GB, 1.33GB/s New Data Upload : 49%|████▉ | 952MB / 1.95GB, 89.4MB/s  ...nts/optim_080000_rank0.pt: 26%|██▌ | 651MB / 2.49GB  ...nts/optim_080000_rank7.pt: 28%|██▊ | 685MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 27%|██▋ | 667MB / 2.49GB  ...nts/optim_080000_rank4.pt: 26%|██▌ | 652MB / 2.49GB  ...nts/optim_080000_rank5.pt: 26%|██▋ | 655MB / 2.49GB  ...nts/optim_080000_rank6.pt: 26%|██▌ | 647MB / 2.49GB  ...nts/optim_080000_rank1.pt: 26%|██▌ | 648MB / 2.49GB  ...nts/optim_080000_rank2.pt: 26%|██▌ | 650MB / 2.49GB  Processing Files (8 / 16) : 56%|█████▌ | 25.0GB / 44.5GB, 1.28GB/s New Data Upload : 49%|████▉ | 989MB / 2.01GB, 92.5MB/s  ...nts/optim_080000_rank0.pt: 27%|██▋ | 669MB / 2.49GB  ...nts/optim_080000_rank7.pt: 28%|██▊ | 701MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 27%|██▋ | 683MB / 2.49GB  ...nts/optim_080000_rank4.pt: 27%|██▋ | 672MB / 2.49GB  ...nts/optim_080000_rank5.pt: 27%|██▋ | 675MB / 2.49GB  ...nts/optim_080000_rank6.pt: 27%|██▋ | 665MB / 2.49GB  ...nts/optim_080000_rank1.pt: 27%|██▋ | 665MB / 2.49GB  ...nts/optim_080000_rank2.pt: 27%|██▋ | 669MB / 2.49GB  Processing Files (8 / 16) : 57%|█████▋ | 25.1GB / 44.5GB, 1.22GB/s New Data Upload : 49%|████▉ | 1.02GB / 2.08GB, 94.8MB/s  ...nts/optim_080000_rank0.pt: 28%|██▊ | 688MB / 2.49GB  ...nts/optim_080000_rank7.pt: 29%|██▉ | 721MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 28%|██▊ | 706MB / 2.49GB  ...nts/optim_080000_rank4.pt: 28%|██▊ | 698MB / 2.49GB  ...nts/optim_080000_rank5.pt: 28%|██▊ | 701MB / 2.49GB  ...nts/optim_080000_rank6.pt: 28%|██▊ | 684MB / 2.49GB  ...nts/optim_080000_rank1.pt: 28%|██▊ | 686MB / 2.49GB  ...nts/optim_080000_rank2.pt: 28%|██▊ | 694MB / 2.49GB  Processing Files (8 / 16) : 57%|█████▋ | 25.3GB / 44.5GB, 1.17GB/s New Data Upload : 50%|█████ | 1.05GB / 2.08GB, 97.8MB/s  ...nts/optim_080000_rank0.pt: 29%|██▉ | 715MB / 2.49GB  ...nts/optim_080000_rank7.pt: 30%|███ | 748MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 30%|██▉ | 735MB / 2.49GB  ...nts/optim_080000_rank4.pt: 29%|██▉ | 730MB / 2.49GB  ...nts/optim_080000_rank5.pt: 29%|██▉ | 733MB / 2.49GB  ...nts/optim_080000_rank6.pt: 29%|██▊ | 711MB / 2.49GB  ...nts/optim_080000_rank1.pt: 29%|██▉ | 715MB / 2.49GB  ...nts/optim_080000_rank2.pt: 29%|██▉ | 725MB / 2.49GB  Processing Files (8 / 16) : 57%|█████▋ | 25.6GB / 44.5GB, 1.12GB/s New Data Upload : 51%|█████ | 1.09GB / 2.15GB, 101MB/s  ...nts/optim_080000_rank0.pt: 30%|██▉ | 736MB / 2.49GB  ...nts/optim_080000_rank7.pt: 31%|███ | 768MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 31%|███ | 759MB / 2.49GB  ...nts/optim_080000_rank4.pt: 30%|███ | 757MB / 2.49GB  ...nts/optim_080000_rank5.pt: 31%|███ | 761MB / 2.49GB  ...nts/optim_080000_rank6.pt: 29%|██▉ | 733MB / 2.49GB  ...nts/optim_080000_rank1.pt: 30%|██▉ | 737MB / 2.49GB  ...nts/optim_080000_rank2.pt: 30%|███ | 753MB / 2.49GB  Processing Files (8 / 16) : 58%|█████▊ | 25.8GB / 44.5GB, 1.07GB/s New Data Upload : 52%|█████▏ | 1.12GB / 2.15GB, 104MB/s  ...nts/optim_080000_rank0.pt: 31%|███ | 763MB / 2.49GB  ...nts/optim_080000_rank7.pt: 32%|███▏ | 795MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 32%|███▏ | 788MB / 2.49GB  ...nts/optim_080000_rank4.pt: 32%|███▏ | 789MB / 2.49GB  ...nts/optim_080000_rank5.pt: 32%|███▏ | 793MB / 2.49GB  ...nts/optim_080000_rank6.pt: 30%|███ | 758MB / 2.49GB  ...nts/optim_080000_rank1.pt: 31%|███ | 765MB / 2.49GB  ...nts/optim_080000_rank2.pt: 32%|███▏ | 785MB / 2.49GB  Processing Files (8 / 16) : 58%|█████▊ | 26.0GB / 44.5GB, 1.06GB/s New Data Upload : 54%|█████▍ | 1.16GB / 2.15GB, 108MB/s  ...nts/optim_080000_rank0.pt: 32%|███▏ | 792MB / 2.49GB  ...nts/optim_080000_rank7.pt: 33%|███▎ | 822MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 33%|███▎ | 813MB / 2.49GB  ...nts/optim_080000_rank4.pt: 33%|███▎ | 817MB / 2.49GB  ...nts/optim_080000_rank5.pt: 33%|███▎ | 821MB / 2.49GB  ...nts/optim_080000_rank6.pt: 32%|███▏ | 785MB / 2.49GB  ...nts/optim_080000_rank1.pt: 32%|███▏ | 791MB / 2.49GB  ...nts/optim_080000_rank2.pt: 33%|███▎ | 813MB / 2.49GB  Processing Files (8 / 16) : 59%|█████▉ | 26.2GB / 44.5GB, 1.05GB/s New Data Upload : 56%|█████▌ | 1.20GB / 2.15GB, 111MB/s  ...nts/optim_080000_rank0.pt: 33%|███▎ | 814MB / 2.49GB  ...nts/optim_080000_rank7.pt: 34%|███▍ | 843MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 33%|███▎ | 831MB / 2.49GB  ...nts/optim_080000_rank4.pt: 34%|███▍ | 839MB / 2.49GB  ...nts/optim_080000_rank5.pt: 34%|███▍ | 843MB / 2.49GB  ...nts/optim_080000_rank6.pt: 32%|███▏ | 803MB / 2.49GB  ...nts/optim_080000_rank1.pt: 33%|███▎ | 811MB / 2.49GB  ...nts/optim_080000_rank2.pt: 34%|███▎ | 833MB / 2.49GB  Processing Files (8 / 16) : 59%|█████▉ | 26.4GB / 44.5GB, 1.04GB/s New Data Upload : 57%|█████▋ | 1.23GB / 2.15GB, 114MB/s  ...nts/optim_080000_rank0.pt: 34%|███▍ | 841MB / 2.49GB  ...nts/optim_080000_rank7.pt: 35%|███▍ | 869MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 34%|███▍ | 854MB / 2.49GB  ...nts/optim_080000_rank4.pt: 35%|███▍ | 868MB / 2.49GB  ...nts/optim_080000_rank5.pt: 35%|███▍ | 868MB / 2.49GB  ...nts/optim_080000_rank6.pt: 33%|███▎ | 824MB / 2.49GB  ...nts/optim_080000_rank1.pt: 34%|███▎ | 836MB / 2.49GB  ...nts/optim_080000_rank2.pt: 35%|███▍ | 859MB / 2.49GB  Processing Files (8 / 16) : 60%|█████▉ | 26.6GB / 44.5GB, 1.03GB/s New Data Upload : 57%|█████▋ | 1.27GB / 2.21GB, 118MB/s  ...nts/optim_080000_rank0.pt: 35%|███▍ | 858MB / 2.49GB  ...nts/optim_080000_rank7.pt: 36%|███▌ | 886MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 35%|███▍ | 869MB / 2.49GB  ...nts/optim_080000_rank4.pt: 36%|███▌ | 887MB / 2.49GB  ...nts/optim_080000_rank5.pt: 36%|███▌ | 887MB / 2.49GB  ...nts/optim_080000_rank6.pt: 34%|███▎ | 838MB / 2.49GB  ...nts/optim_080000_rank1.pt: 34%|███▍ | 852MB / 2.49GB  ...nts/optim_080000_rank2.pt: 35%|███▌ | 876MB / 2.49GB  Processing Files (8 / 16) : 60%|██████ | 26.7GB / 44.5GB, 1.02GB/s New Data Upload : 58%|█████▊ | 1.29GB / 2.21GB, 120MB/s  ...nts/optim_080000_rank0.pt: 36%|███▌ | 886MB / 2.49GB  ...nts/optim_080000_rank7.pt: 37%|███▋ | 910MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 36%|███▌ | 892MB / 2.49GB  ...nts/optim_080000_rank4.pt: 37%|███▋ | 914MB / 2.49GB  ...nts/optim_080000_rank5.pt: 37%|███▋ | 911MB / 2.49GB  ...nts/optim_080000_rank6.pt: 35%|███▍ | 861MB / 2.49GB  ...nts/optim_080000_rank1.pt: 35%|███▌ | 875MB / 2.49GB  ...nts/optim_080000_rank2.pt: 36%|███▌ | 900MB / 2.49GB  Processing Files (8 / 16) : 60%|██████ | 26.9GB / 44.5GB, 1.01GB/s New Data Upload : 60%|██████ | 1.33GB / 2.21GB, 124MB/s  ...nts/optim_080000_rank0.pt: 37%|███▋ | 911MB / 2.49GB  ...nts/optim_080000_rank7.pt: 38%|███▊ | 933MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 37%|███▋ | 915MB / 2.49GB  ...nts/optim_080000_rank4.pt: 38%|███▊ | 941MB / 2.49GB  ...nts/optim_080000_rank5.pt: 38%|███▊ | 936MB / 2.49GB  ...nts/optim_080000_rank6.pt: 36%|███▌ | 883MB / 2.49GB  ...nts/optim_080000_rank1.pt: 36%|███▌ | 898MB / 2.49GB  ...nts/optim_080000_rank2.pt: 37%|███▋ | 925MB / 2.49GB  Processing Files (8 / 16) : 61%|██████ | 27.1GB / 44.5GB, 1.01GB/s New Data Upload : 60%|█████▉ | 1.37GB / 2.28GB, 127MB/s  ...nts/optim_080000_rank0.pt: 38%|███▊ | 936MB / 2.49GB  ...nts/optim_080000_rank7.pt: 39%|███▊ | 958MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 38%|███▊ | 939MB / 2.49GB  ...nts/optim_080000_rank4.pt: 39%|███▉ | 967MB / 2.49GB  ...nts/optim_080000_rank5.pt: 39%|███▊ | 962MB / 2.49GB  ...nts/optim_080000_rank6.pt: 36%|███▋ | 904MB / 2.49GB  ...nts/optim_080000_rank1.pt: 37%|███▋ | 922MB / 2.49GB  ...nts/optim_080000_rank2.pt: 38%|███▊ | 950MB / 2.49GB  Processing Files (8 / 16) : 61%|██████▏ | 27.3GB / 44.5GB, 1.00GB/s New Data Upload : 60%|█████▉ | 1.40GB / 2.35GB, 131MB/s  ...nts/optim_080000_rank0.pt: 39%|███▉ | 971MB / 2.49GB  ...nts/optim_080000_rank7.pt: 40%|███▉ | 992MB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 39%|███▉ | 971MB / 2.49GB  ...nts/optim_080000_rank4.pt: 40%|████ | 1.00GB / 2.49GB  ...nts/optim_080000_rank5.pt: 40%|████ | 996MB / 2.49GB  ...nts/optim_080000_rank6.pt: 38%|███▊ | 935MB / 2.49GB  ...nts/optim_080000_rank1.pt: 39%|███▊ | 958MB / 2.49GB  ...nts/optim_080000_rank2.pt: 40%|███▉ | 983MB / 2.49GB  Processing Files (8 / 16) : 62%|██████▏ | 27.6GB / 44.5GB, 1.01GB/s New Data Upload : 60%|██████ | 1.45GB / 2.41GB, 136MB/s  ...nts/optim_080000_rank0.pt: 40%|████ | 1.00GB / 2.49GB  ...nts/optim_080000_rank7.pt: 41%|████ | 1.02GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 40%|████ | 1.00GB / 2.49GB  ...nts/optim_080000_rank4.pt: 42%|████▏ | 1.03GB / 2.49GB  ...nts/optim_080000_rank5.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_080000_rank6.pt: 39%|███▉ | 964MB / 2.49GB  ...nts/optim_080000_rank1.pt: 40%|███▉ | 989MB / 2.49GB  ...nts/optim_080000_rank2.pt: 41%|████ | 1.01GB / 2.49GB  Processing Files (8 / 16) : 63%|██████▎ | 27.8GB / 44.5GB, 1.00GB/s New Data Upload : 60%|██████ | 1.49GB / 2.48GB, 139MB/s  ...nts/optim_080000_rank0.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_080000_rank7.pt: 42%|████▏ | 1.05GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_080000_rank4.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_080000_rank5.pt: 42%|████▏ | 1.06GB / 2.49GB  ...nts/optim_080000_rank6.pt: 40%|███▉ | 991MB / 2.49GB  ...nts/optim_080000_rank1.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_080000_rank2.pt: 42%|████▏ | 1.04GB / 2.49GB  Processing Files (8 / 16) : 63%|██████▎ | 28.0GB / 44.5GB, 1.00GB/s New Data Upload : 58%|█████▊ | 1.53GB / 2.62GB, 143MB/s  ...nts/optim_080000_rank0.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_080000_rank7.pt: 43%|████▎ | 1.07GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_080000_rank4.pt: 43%|████▎ | 1.08GB / 2.49GB  ...nts/optim_080000_rank5.pt: 43%|████▎ | 1.08GB / 2.49GB  ...nts/optim_080000_rank6.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_080000_rank1.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_080000_rank2.pt: 43%|████▎ | 1.06GB / 2.49GB  Processing Files (8 / 16) : 63%|██████▎ | 28.2GB / 44.5GB, 995MB/s New Data Upload : 58%|█████▊ | 1.56GB / 2.68GB, 146MB/s  ...nts/optim_080000_rank0.pt: 44%|████▎ | 1.09GB / 2.49GB  ...nts/optim_080000_rank7.pt: 44%|████▍ | 1.10GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 44%|████▍ | 1.09GB / 2.49GB  ...nts/optim_080000_rank4.pt: 45%|████▍ | 1.12GB / 2.49GB  ...nts/optim_080000_rank5.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_080000_rank6.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_080000_rank1.pt: 43%|████▎ | 1.08GB / 2.49GB  ...nts/optim_080000_rank2.pt: 44%|████▍ | 1.10GB / 2.49GB  Processing Files (8 / 16) : 64%|██████▍ | 28.5GB / 44.5GB, 993MB/s New Data Upload : 58%|█████▊ | 1.61GB / 2.75GB, 150MB/s  ...nts/optim_080000_rank0.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_080000_rank7.pt: 45%|████▌ | 1.12GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_080000_rank4.pt: 46%|████▌ | 1.13GB / 2.49GB  ...nts/optim_080000_rank5.pt: 46%|████▌ | 1.13GB / 2.49GB  ...nts/optim_080000_rank6.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_080000_rank1.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_080000_rank2.pt: 45%|████▌ | 1.12GB / 2.49GB  Processing Files (8 / 16) : 64%|██████▍ | 28.6GB / 44.5GB, 978MB/s New Data Upload : 59%|█████▉ | 1.63GB / 2.75GB, 152MB/s  ...nts/optim_080000_rank0.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_080000_rank7.pt: 46%|████▋ | 1.16GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_080000_rank4.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_080000_rank5.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_080000_rank6.pt: 44%|████▍ | 1.11GB / 2.49GB  ...nts/optim_080000_rank1.pt: 46%|████▌ | 1.13GB / 2.49GB  ...nts/optim_080000_rank2.pt: 46%|████▋ | 1.15GB / 2.49GB  Processing Files (8 / 16) : 65%|██████▍ | 28.9GB / 44.5GB, 974MB/s New Data Upload : 59%|█████▉ | 1.67GB / 2.82GB, 155MB/s  ...nts/optim_080000_rank0.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_080000_rank7.pt: 48%|████▊ | 1.18GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_080000_rank4.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_080000_rank5.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_080000_rank6.pt: 46%|████▌ | 1.13GB / 2.49GB  ...nts/optim_080000_rank1.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_080000_rank2.pt: 48%|████▊ | 1.18GB / 2.49GB  Processing Files (8 / 16) : 65%|██████▌ | 29.1GB / 44.5GB, 963MB/s New Data Upload : 58%|█████▊ | 1.71GB / 2.95GB, 158MB/s  ...nts/optim_080000_rank0.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_080000_rank7.pt: 49%|████▉ | 1.21GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_080000_rank4.pt: 49%|████▉ | 1.23GB / 2.49GB  ...nts/optim_080000_rank5.pt: 49%|████▉ | 1.23GB / 2.49GB  ...nts/optim_080000_rank6.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_080000_rank1.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_080000_rank2.pt: 49%|████▉ | 1.21GB / 2.49GB  Processing Files (8 / 16) : 66%|██████▌ | 29.4GB / 44.5GB, 957MB/s New Data Upload : 56%|█████▌ | 1.76GB / 3.15GB, 161MB/s  ...nts/optim_080000_rank0.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_080000_rank7.pt: 50%|████▉ | 1.24GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_080000_rank4.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_080000_rank5.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_080000_rank6.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_080000_rank1.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_080000_rank2.pt: 50%|████▉ | 1.24GB / 2.49GB  Processing Files (8 / 16) : 67%|██████▋ | 29.6GB / 44.5GB, 941MB/s New Data Upload : 57%|█████▋ | 1.79GB / 3.15GB, 162MB/s  ...nts/optim_080000_rank0.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_080000_rank7.pt: 51%|█████ | 1.27GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_080000_rank4.pt: 52%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_080000_rank5.pt: 52%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_080000_rank6.pt: 49%|████▉ | 1.23GB / 2.49GB  ...nts/optim_080000_rank1.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_080000_rank2.pt: 51%|█████ | 1.27GB / 2.49GB  Processing Files (8 / 16) : 67%|██████▋ | 29.8GB / 44.5GB, 941MB/s New Data Upload : 57%|█████▋ | 1.84GB / 3.22GB, 164MB/s  ...nts/optim_080000_rank0.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_080000_rank7.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_080000_rank4.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_080000_rank5.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_080000_rank6.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_080000_rank1.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_080000_rank2.pt: 52%|█████▏ | 1.30GB / 2.49GB  Processing Files (8 / 16) : 68%|██████▊ | 30.1GB / 44.5GB, 945MB/s New Data Upload : 59%|█████▊ | 1.89GB / 3.22GB, 167MB/s  ...nts/optim_080000_rank0.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_080000_rank7.pt: 54%|█████▎ | 1.33GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_080000_rank4.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_080000_rank5.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_080000_rank6.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_080000_rank1.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_080000_rank2.pt: 54%|█████▎ | 1.33GB / 2.49GB  Processing Files (8 / 16) : 68%|██████▊ | 30.4GB / 44.5GB, 956MB/s New Data Upload : 59%|█████▉ | 1.93GB / 3.29GB, 168MB/s  ...nts/optim_080000_rank0.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_080000_rank7.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_080000_rank4.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_080000_rank5.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_080000_rank6.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_080000_rank1.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_080000_rank2.pt: 55%|█████▍ | 1.36GB / 2.49GB  Processing Files (8 / 16) : 69%|██████▉ | 30.6GB / 44.5GB, 956MB/s New Data Upload : 59%|█████▉ | 1.97GB / 3.35GB, 169MB/s  ...nts/optim_080000_rank0.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_080000_rank7.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_080000_rank4.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_080000_rank5.pt: 56%|█████▌ | 1.40GB / 2.49GB  ...nts/optim_080000_rank6.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_080000_rank1.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_080000_rank2.pt: 56%|█████▌ | 1.39GB / 2.49GB  Processing Files (8 / 16) : 69%|██████▉ | 30.8GB / 44.5GB, 959MB/s New Data Upload : 60%|██████ | 2.01GB / 3.35GB, 170MB/s  ...nts/optim_080000_rank0.pt: 56%|█████▌ | 1.40GB / 2.49GB  ...nts/optim_080000_rank7.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_080000_rank4.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_080000_rank5.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_080000_rank6.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_080000_rank1.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_080000_rank2.pt: 57%|█████▋ | 1.41GB / 2.49GB  Processing Files (8 / 16) : 70%|██████▉ | 31.0GB / 44.5GB, 963MB/s New Data Upload : 60%|█████▉ | 2.05GB / 3.42GB, 170MB/s  ...nts/optim_080000_rank0.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_080000_rank7.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 57%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_080000_rank4.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_080000_rank5.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_080000_rank6.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_080000_rank1.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_080000_rank2.pt: 57%|█████▋ | 1.43GB / 2.49GB  Processing Files (8 / 16) : 70%|██████▉ | 31.1GB / 44.5GB, 966MB/s New Data Upload : 61%|██████ | 2.07GB / 3.42GB, 170MB/s  ...nts/optim_080000_rank0.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_080000_rank7.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 58%|█████▊ | 1.43GB / 2.49GB  ...nts/optim_080000_rank4.pt: 59%|█████▉ | 1.46GB / 2.49GB  ...nts/optim_080000_rank5.pt: 59%|█████▉ | 1.46GB / 2.49GB  ...nts/optim_080000_rank6.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_080000_rank1.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_080000_rank2.pt: 59%|█████▊ | 1.45GB / 2.49GB  Processing Files (8 / 16) : 70%|███████ | 31.3GB / 44.5GB, 976MB/s New Data Upload : 62%|██████▏ | 2.11GB / 3.42GB, 171MB/s  ...nts/optim_080000_rank0.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_080000_rank7.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_080000_rank4.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_080000_rank5.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_080000_rank6.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_080000_rank1.pt: 59%|█████▉ | 1.46GB / 2.49GB  ...nts/optim_080000_rank2.pt: 59%|█████▉ | 1.47GB / 2.49GB  Processing Files (8 / 16) : 71%|███████ | 31.5GB / 44.5GB, 973MB/s New Data Upload : 61%|██████▏ | 2.14GB / 3.49GB, 170MB/s  ...nts/optim_080000_rank0.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 60%|██████ | 1.50GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_080000_rank4.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_080000_rank5.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_080000_rank6.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_080000_rank1.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 60%|██████ | 1.50GB / 2.49GB  Processing Files (8 / 16) : 71%|███████▏ | 31.7GB / 44.5GB, 980MB/s New Data Upload : 61%|██████▏ | 2.18GB / 3.55GB, 171MB/s  ...nts/optim_080000_rank0.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_080000_rank7.pt: 61%|██████ | 1.52GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 60%|██████ | 1.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 61%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_080000_rank5.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_080000_rank6.pt: 60%|██████ | 1.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_080000_rank2.pt: 61%|██████ | 1.52GB / 2.49GB  Processing Files (8 / 16) : 72%|███████▏ | 31.9GB / 44.5GB, 979MB/s New Data Upload : 62%|██████▏ | 2.22GB / 3.59GB, 172MB/s  ...nts/optim_080000_rank0.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_080000_rank7.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 61%|██████ | 1.52GB / 2.49GB  ...nts/optim_080000_rank4.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_080000_rank5.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_080000_rank6.pt: 61%|██████▏ | 1.52GB / 2.49GB  ...nts/optim_080000_rank1.pt: 62%|██████▏ | 1.54GB / 2.49GB  [+ 1 files] : 33%|███▎ | 1.62GB / 4.97GB  Processing Files (8 / 17) : 72%|███████▏ | 32.2GB / 44.5GB, 989MB/s New Data Upload : 63%|██████▎ | 2.26GB / 3.59GB, 173MB/s  ...nts/optim_080000_rank0.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_080000_rank7.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_080000_rank4.pt: 64%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_080000_rank5.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_080000_rank6.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_080000_rank1.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_080000_rank2.pt: 63%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_075000_rank0.pt: 9%|▉ | 232MB / 2.49GB  Processing Files (8 / 17) : 73%|███████▎ | 32.5GB / 44.5GB, 1.01GB/s New Data Upload : 64%|██████▍ | 2.30GB / 3.59GB, 174MB/s  ...nts/optim_080000_rank0.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_080000_rank7.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_080000_rank4.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_080000_rank5.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_080000_rank6.pt: 64%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_080000_rank1.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_080000_rank2.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_075000_rank0.pt: 15%|█▌ | 384MB / 2.49GB  Processing Files (8 / 17) : 74%|███████▍ | 32.9GB / 44.5GB, 1.03GB/s New Data Upload : 65%|██████▍ | 2.33GB / 3.59GB, 175MB/s  ...nts/optim_080000_rank0.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_080000_rank7.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_080000_rank4.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_080000_rank5.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_080000_rank6.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_080000_rank1.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_080000_rank2.pt: 65%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_075000_rank0.pt: 21%|██ | 512MB / 2.49GB  Processing Files (8 / 17) : 75%|███████▍ | 33.2GB / 44.5GB, 1.04GB/s New Data Upload : 65%|██████▍ | 2.37GB / 3.66GB, 175MB/s  ...nts/optim_080000_rank0.pt: 66%|██████▌ | 1.65GB / 2.49GB  ...nts/optim_080000_rank7.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_080000_rank4.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_080000_rank5.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_080000_rank6.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_080000_rank1.pt: 66%|██████▌ | 1.65GB / 2.49GB  ...nts/optim_080000_rank2.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_075000_rank0.pt: 27%|██▋ | 664MB / 2.49GB  Processing Files (8 / 17) : 76%|███████▌ | 33.6GB / 44.5GB, 1.06GB/s New Data Upload : 66%|██████▌ | 2.42GB / 3.66GB, 177MB/s  ...nts/optim_080000_rank0.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_080000_rank7.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 67%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_080000_rank4.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_080000_rank5.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_080000_rank6.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_080000_rank1.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_080000_rank2.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_075000_rank0.pt: 33%|███▎ | 816MB / 2.49GB  Processing Files (8 / 17) : 77%|███████▋ | 34.1GB / 44.5GB, 1.09GB/s New Data Upload : 66%|██████▋ | 2.47GB / 3.72GB, 180MB/s  ...nts/optim_080000_rank0.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_080000_rank7.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_080000_rank4.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_080000_rank5.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_080000_rank6.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_080000_rank1.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_080000_rank2.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_075000_rank0.pt: 39%|███▉ | 968MB / 2.49GB  Processing Files (8 / 17) : 77%|███████▋ | 34.4GB / 44.5GB, 1.11GB/s New Data Upload : 67%|██████▋ | 2.51GB / 3.72GB, 181MB/s  ...nts/optim_080000_rank0.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_080000_rank7.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_080000_rank4.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_080000_rank5.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_080000_rank6.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_080000_rank1.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_080000_rank2.pt: 70%|███████ | 1.75GB / 2.49GB  [+ 1 files] : 24%|██▎ | 1.14GB / 4.83GB  Processing Files (8 / 18) : 78%|███████▊ | 34.9GB / 44.5GB, 1.13GB/s New Data Upload : 69%|██████▊ | 2.55GB / 3.72GB, 182MB/s  ...nts/optim_080000_rank0.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_080000_rank7.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_080000_rank3.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_080000_rank4.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_080000_rank5.pt: 73%|███████▎ | 1.80GB / 2.49GB  ...nts/optim_080000_rank6.pt: 71%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_080000_rank1.pt: 71%|███████▏ | 1.77GB / 2.49GB  ...nts/optim_080000_rank2.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_075000_rank0.pt: 51%|█████ | 1.26GB / 2.49GB  ...eckpoints/model_080000.pt: 0%| | 7.15MB / 2.35GB  Processing Files (8 / 18) : 79%|███████▉ | 35.3GB / 44.5GB, 1.15GB/s New Data Upload : 69%|██████▊ | 2.60GB / 3.79GB, 184MB/s  ...nts/optim_080000_rank0.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_080000_rank7.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_080000_rank3.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_080000_rank4.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_080000_rank5.pt: 74%|███████▍ | 1.83GB / 2.49GB  ...nts/optim_080000_rank6.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_080000_rank1.pt: 73%|███████▎ | 1.80GB / 2.49GB  ...nts/optim_080000_rank2.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_075000_rank0.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...eckpoints/model_080000.pt: 0%| | 7.15MB / 2.35GB  Processing Files (8 / 18) : 80%|████████ | 35.7GB / 44.5GB, 1.16GB/s New Data Upload : 70%|██████▉ | 2.65GB / 3.79GB, 184MB/s  ...nts/optim_080000_rank0.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_080000_rank7.pt: 75%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_080000_rank3.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_080000_rank4.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_080000_rank5.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_080000_rank6.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_080000_rank1.pt: 74%|███████▍ | 1.83GB / 2.49GB  ...nts/optim_080000_rank2.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_075000_rank0.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...eckpoints/model_080000.pt: 0%| | 8.89MB / 2.35GB  Processing Files (8 / 18) : 81%|████████ | 36.1GB / 44.5GB, 1.19GB/s New Data Upload : 71%|███████ | 2.70GB / 3.79GB, 186MB/s  ...nts/optim_080000_rank0.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_080000_rank7.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_080000_rank3.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_080000_rank4.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_080000_rank5.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_080000_rank6.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_080000_rank1.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_080000_rank2.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_075000_rank0.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...eckpoints/model_080000.pt: 0%| | 10.6MB / 2.35GB  Processing Files (8 / 18) : 82%|████████▏ | 36.5GB / 44.5GB, 1.22GB/s New Data Upload : 73%|███████▎ | 2.75GB / 3.79GB, 188MB/s  ...nts/optim_080000_rank0.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_080000_rank7.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_080000_rank3.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_080000_rank4.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_080000_rank5.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_080000_rank6.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_080000_rank1.pt: 77%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_080000_rank2.pt: 77%|███████▋ | 1.93GB / 2.49GB  ...nts/optim_075000_rank0.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...eckpoints/model_080000.pt: 0%| | 11.2MB / 2.35GB  Processing Files (8 / 18) : 83%|████████▎ | 37.0GB / 44.5GB, 1.25GB/s New Data Upload : 73%|███████▎ | 2.80GB / 3.86GB, 192MB/s  ...nts/optim_080000_rank0.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_080000_rank7.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_080000_rank3.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_080000_rank4.pt: 79%|███████▊ | 1.96GB / 2.49GB  ...nts/optim_080000_rank5.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_080000_rank6.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_080000_rank1.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_080000_rank2.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_075000_rank0.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...eckpoints/model_080000.pt: 0%| | 11.2MB / 2.35GB  Processing Files (8 / 18) : 84%|████████▍ | 37.3GB / 44.5GB, 1.27GB/s New Data Upload : 74%|███████▎ | 2.84GB / 3.86GB, 194MB/s  ...nts/optim_080000_rank0.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_080000_rank7.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_080000_rank3.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_080000_rank4.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_080000_rank5.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_080000_rank6.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_080000_rank1.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_080000_rank2.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_075000_rank0.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...eckpoints/model_080000.pt: 1%| | 15.2MB / 2.35GB  Processing Files (8 / 18) : 85%|████████▍ | 37.7GB / 44.5GB, 1.30GB/s New Data Upload : 74%|███████▎ | 2.89GB / 3.92GB, 196MB/s  ...nts/optim_080000_rank0.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_080000_rank7.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_080000_rank3.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_080000_rank4.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_080000_rank5.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_080000_rank6.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_080000_rank1.pt: 80%|███████▉ | 1.99GB / 2.49GB  ...nts/optim_080000_rank2.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_075000_rank0.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...eckpoints/model_080000.pt: 1%| | 20.8MB / 2.35GB  Processing Files (8 / 18) : 86%|████████▌ | 38.1GB / 44.5GB, 1.32GB/s New Data Upload : 75%|███████▌ | 2.95GB / 3.92GB, 199MB/s  ...nts/optim_080000_rank0.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_080000_rank7.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_080000_rank3.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_080000_rank4.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_080000_rank5.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_080000_rank6.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_080000_rank1.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_080000_rank2.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_075000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...eckpoints/model_080000.pt: 1%| | 23.6MB / 2.35GB  Processing Files (8 / 18) : 86%|████████▋ | 38.4GB / 44.5GB, 1.33GB/s New Data Upload : 75%|███████▍ | 2.98GB / 3.99GB, 199MB/s  ...nts/optim_080000_rank0.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_080000_rank7.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_080000_rank3.pt: 81%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_080000_rank4.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_080000_rank5.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_080000_rank6.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_080000_rank1.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_080000_rank2.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 1%|▏ | 29.8MB / 2.35GB  Processing Files (9 / 18) : 87%|████████▋ | 38.7GB / 44.5GB, 1.34GB/s New Data Upload : 76%|███████▌ | 3.03GB / 3.99GB, 200MB/s  ...nts/optim_080000_rank0.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_080000_rank7.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_080000_rank3.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_080000_rank4.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_080000_rank5.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_080000_rank6.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_080000_rank1.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_080000_rank2.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 2%|▏ | 38.1MB / 2.35GB  Processing Files (9 / 18) : 87%|████████▋ | 38.9GB / 44.5GB, 1.35GB/s New Data Upload : 76%|███████▌ | 3.07GB / 4.06GB, 201MB/s  ...nts/optim_080000_rank0.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_080000_rank7.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_080000_rank3.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_080000_rank4.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_080000_rank5.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_080000_rank6.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_080000_rank1.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_080000_rank2.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 2%|▏ | 48.7MB / 2.35GB  Processing Files (9 / 18) : 88%|████████▊ | 39.1GB / 44.5GB, 1.35GB/s New Data Upload : 76%|███████▌ | 3.12GB / 4.12GB, 203MB/s  ...nts/optim_080000_rank0.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_080000_rank7.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_080000_rank3.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_080000_rank4.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_080000_rank5.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_080000_rank6.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_080000_rank1.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_080000_rank2.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 2%|▏ | 55.9MB / 2.35GB  Processing Files (9 / 18) : 88%|████████▊ | 39.2GB / 44.5GB, 1.34GB/s New Data Upload : 76%|███████▋ | 3.15GB / 4.12GB, 202MB/s  ...nts/optim_080000_rank0.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_080000_rank7.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_080000_rank3.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_080000_rank4.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_080000_rank5.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_080000_rank6.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_080000_rank1.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_080000_rank2.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 3%|▎ | 63.8MB / 2.35GB  Processing Files (9 / 18) : 89%|████████▊ | 39.4GB / 44.5GB, 1.33GB/s New Data Upload : 77%|███████▋ | 3.19GB / 4.12GB, 202MB/s  ...nts/optim_080000_rank0.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_080000_rank7.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_080000_rank3.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_080000_rank4.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_080000_rank5.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_080000_rank6.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_080000_rank1.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_080000_rank2.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 3%|▎ | 73.8MB / 2.35GB  Processing Files (9 / 18) : 89%|████████▉ | 39.5GB / 44.5GB, 1.33GB/s New Data Upload : 77%|███████▋ | 3.23GB / 4.19GB, 202MB/s  ...nts/optim_080000_rank0.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_080000_rank7.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_080000_rank3.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_080000_rank4.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_080000_rank5.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_080000_rank6.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_080000_rank1.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_080000_rank2.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 4%|▎ | 82.7MB / 2.35GB  Processing Files (9 / 18) : 89%|████████▉ | 39.7GB / 44.5GB, 1.32GB/s New Data Upload : 78%|███████▊ | 3.27GB / 4.19GB, 202MB/s  ...nts/optim_080000_rank0.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_080000_rank7.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_080000_rank3.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_080000_rank4.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_080000_rank5.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_080000_rank6.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_080000_rank1.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_080000_rank2.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 4%|▍ | 96.6MB / 2.35GB  Processing Files (9 / 18) : 89%|████████▉ | 39.8GB / 44.5GB, 1.32GB/s New Data Upload : 79%|███████▊ | 3.30GB / 4.19GB, 203MB/s  ...nts/optim_080000_rank0.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_080000_rank7.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_080000_rank3.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_080000_rank4.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_080000_rank5.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_080000_rank6.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_080000_rank1.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_080000_rank2.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 5%|▍ | 107MB / 2.35GB  Processing Files (9 / 18) : 90%|████████▉ | 40.0GB / 44.5GB, 1.31GB/s New Data Upload : 80%|███████▉ | 3.34GB / 4.19GB, 203MB/s  ...nts/optim_080000_rank0.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_080000_rank7.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_080000_rank3.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_080000_rank4.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_080000_rank5.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_080000_rank6.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_080000_rank1.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_080000_rank2.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 5%|▌ | 121MB / 2.35GB  Processing Files (9 / 18) : 90%|█████████ | 40.1GB / 44.5GB, 1.31GB/s New Data Upload : 79%|███████▉ | 3.37GB / 4.26GB, 204MB/s  ...nts/optim_080000_rank0.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_080000_rank7.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_080000_rank3.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_080000_rank4.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_080000_rank5.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_080000_rank6.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_080000_rank1.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_080000_rank2.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 6%|▌ | 141MB / 2.35GB  Processing Files (9 / 18) : 90%|█████████ | 40.2GB / 44.5GB, 1.31GB/s New Data Upload : 80%|████████ | 3.42GB / 4.26GB, 205MB/s  ...nts/optim_080000_rank0.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_080000_rank7.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_080000_rank3.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_080000_rank4.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_080000_rank5.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_080000_rank6.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_080000_rank1.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_080000_rank2.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 7%|▋ | 159MB / 2.35GB  Processing Files (9 / 18) : 91%|█████████ | 40.3GB / 44.5GB, 1.30GB/s New Data Upload : 80%|███████▉ | 3.45GB / 4.32GB, 204MB/s  ...nts/optim_080000_rank0.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_080000_rank7.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_080000_rank3.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_080000_rank4.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_080000_rank5.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_080000_rank6.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_080000_rank1.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_080000_rank2.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 8%|▊ | 182MB / 2.35GB  Processing Files (9 / 18) : 91%|█████████ | 40.5GB / 44.5GB, 1.29GB/s New Data Upload : 81%|████████ | 3.49GB / 4.32GB, 205MB/s  ...nts/optim_080000_rank0.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_080000_rank7.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_080000_rank3.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_080000_rank4.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_080000_rank5.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_080000_rank6.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_080000_rank1.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_080000_rank2.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 9%|▊ | 201MB / 2.35GB  Processing Files (9 / 18) : 91%|█████████ | 40.6GB / 44.5GB, 1.28GB/s New Data Upload : 80%|████████ | 3.53GB / 4.39GB, 204MB/s  ...nts/optim_080000_rank0.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_080000_rank7.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_080000_rank3.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_080000_rank4.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_080000_rank5.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_080000_rank6.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_080000_rank1.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_080000_rank2.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 9%|▉ | 219MB / 2.35GB  Processing Files (9 / 18) : 91%|█████████▏| 40.7GB / 44.5GB, 1.26GB/s New Data Upload : 80%|███████▉ | 3.56GB / 4.46GB, 203MB/s  ...nts/optim_080000_rank0.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_080000_rank7.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_080000_rank3.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_080000_rank4.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_080000_rank5.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_080000_rank6.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_080000_rank1.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_080000_rank2.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 10%|█ | 238MB / 2.35GB  Processing Files (9 / 18) : 92%|█████████▏| 40.8GB / 44.5GB, 1.25GB/s New Data Upload : 79%|███████▉ | 3.60GB / 4.53GB, 203MB/s  ...nts/optim_080000_rank0.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_080000_rank7.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_080000_rank3.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_080000_rank4.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_080000_rank5.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_080000_rank6.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_080000_rank1.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_080000_rank2.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 11%|█ | 260MB / 2.35GB  Processing Files (9 / 18) : 92%|█████████▏| 40.9GB / 44.5GB, 1.24GB/s New Data Upload : 80%|████████ | 3.63GB / 4.53GB, 203MB/s  ...nts/optim_080000_rank0.pt: 94%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_080000_rank7.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_080000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_080000_rank4.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_080000_rank5.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_080000_rank6.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_080000_rank1.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_080000_rank2.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 12%|█▏ | 283MB / 2.35GB  Processing Files (9 / 18) : 92%|█████████▏| 40.9GB / 44.5GB, 1.22GB/s New Data Upload : 80%|███████▉ | 3.66GB / 4.59GB, 202MB/s  ...nts/optim_080000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_080000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_080000_rank3.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_080000_rank4.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_080000_rank5.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_080000_rank6.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_080000_rank1.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_080000_rank2.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 13%|█▎ | 305MB / 2.35GB  Processing Files (9 / 18) : 92%|█████████▏| 41.1GB / 44.5GB, 1.22GB/s New Data Upload : 79%|███████▉ | 3.70GB / 4.66GB, 203MB/s  ...nts/optim_080000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_080000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_080000_rank3.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_080000_rank4.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_080000_rank5.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_080000_rank6.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_080000_rank1.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_080000_rank2.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 14%|█▍ | 329MB / 2.35GB  Processing Files (9 / 18) : 92%|█████████▏| 41.1GB / 44.5GB, 1.20GB/s New Data Upload : 80%|████████ | 3.73GB / 4.66GB, 202MB/s  ...nts/optim_080000_rank0.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_080000_rank7.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_080000_rank3.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_080000_rank4.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_080000_rank5.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_080000_rank6.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_080000_rank1.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_080000_rank2.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 15%|█▌ | 357MB / 2.35GB  Processing Files (9 / 18) : 93%|█████████▎| 41.2GB / 44.5GB, 1.18GB/s New Data Upload : 80%|███████▉ | 3.77GB / 4.73GB, 202MB/s  ...nts/optim_080000_rank0.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_080000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_080000_rank3.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_080000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_080000_rank5.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_080000_rank6.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_080000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_080000_rank2.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 17%|█▋ | 390MB / 2.35GB  Processing Files (9 / 18) : 93%|█████████▎| 41.3GB / 44.5GB, 1.17GB/s New Data Upload : 80%|███████▉ | 3.82GB / 4.79GB, 202MB/s  ...nts/optim_080000_rank0.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_080000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_080000_rank3.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_080000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_080000_rank5.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_080000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_080000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_080000_rank2.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 18%|█▊ | 422MB / 2.35GB  Processing Files (9 / 18) : 93%|█████████▎| 41.4GB / 44.5GB, 1.16GB/s New Data Upload : 79%|███████▉ | 3.86GB / 4.86GB, 202MB/s  ...nts/optim_080000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_080000_rank7.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_080000_rank3.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_080000_rank4.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_080000_rank5.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_080000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_080000_rank1.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_080000_rank2.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 20%|█▉ | 461MB / 2.35GB  Processing Files (9 / 18) : 93%|█████████▎| 41.5GB / 44.5GB, 1.14GB/s New Data Upload : 80%|████████ | 3.91GB / 4.86GB, 203MB/s  ...nts/optim_080000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_080000_rank7.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_080000_rank3.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_080000_rank4.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_080000_rank5.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_080000_rank6.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_080000_rank1.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_080000_rank2.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 20%|██ | 480MB / 2.35GB  Processing Files (9 / 18) : 93%|█████████▎| 41.5GB / 44.5GB, 1.12GB/s New Data Upload : 81%|████████ | 3.93GB / 4.86GB, 200MB/s  ...nts/optim_080000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_080000_rank7.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_080000_rank3.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_080000_rank4.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_080000_rank5.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_080000_rank6.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_080000_rank1.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_080000_rank2.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 22%|██▏ | 506MB / 2.35GB  Processing Files (9 / 18) : 94%|█████████▎| 41.6GB / 44.5GB, 1.10GB/s New Data Upload : 80%|████████ | 3.97GB / 4.93GB, 200MB/s  ...nts/optim_080000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_080000_rank7.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_080000_rank3.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_080000_rank4.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_080000_rank5.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_080000_rank6.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_080000_rank1.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_080000_rank2.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 23%|██▎ | 541MB / 2.35GB  Processing Files (9 / 18) : 94%|█████████▎| 41.7GB / 44.5GB, 1.09GB/s New Data Upload : 80%|████████ | 4.01GB / 5.00GB, 200MB/s  ...nts/optim_080000_rank0.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_080000_rank7.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_080000_rank3.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_080000_rank4.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_080000_rank5.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_080000_rank6.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_080000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_080000_rank2.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 24%|██▍ | 571MB / 2.35GB  Processing Files (9 / 18) : 94%|█████████▍| 41.8GB / 44.5GB, 1.07GB/s New Data Upload : 81%|████████ | 4.05GB / 5.00GB, 200MB/s  ...nts/optim_080000_rank0.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_080000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_080000_rank3.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_080000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_080000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_080000_rank6.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_080000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_080000_rank2.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 26%|██▌ | 600MB / 2.35GB  Processing Files (9 / 18) : 94%|█████████▍| 41.8GB / 44.5GB, 1.06GB/s New Data Upload : 81%|████████ | 4.09GB / 5.06GB, 200MB/s  ...nts/optim_080000_rank0.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_080000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_080000_rank3.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_080000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_080000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_080000_rank6.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_080000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_080000_rank2.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 27%|██▋ | 629MB / 2.35GB  Processing Files (9 / 18) : 94%|█████████▍| 41.9GB / 44.5GB, 1.06GB/s New Data Upload : 80%|████████ | 4.12GB / 5.13GB, 201MB/s  ...nts/optim_080000_rank0.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_080000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_080000_rank3.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_080000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_080000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_080000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_080000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_080000_rank2.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 28%|██▊ | 659MB / 2.35GB  Processing Files (9 / 18) : 94%|█████████▍| 41.9GB / 44.5GB, 1.04GB/s New Data Upload : 81%|████████ | 4.16GB / 5.13GB, 200MB/s  ...nts/optim_080000_rank0.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_080000_rank7.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_080000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_080000_rank4.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_080000_rank5.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_080000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_080000_rank1.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_080000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 29%|██▉ | 683MB / 2.35GB  Processing Files (9 / 18) : 94%|█████████▍| 42.0GB / 44.5GB, 1.03GB/s New Data Upload : 81%|████████ | 4.19GB / 5.20GB, 201MB/s  ...nts/optim_080000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank7.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_080000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_080000_rank4.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_080000_rank5.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_080000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_080000_rank1.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_080000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 30%|███ | 708MB / 2.35GB  Processing Files (9 / 18) : 95%|█████████▍| 42.1GB / 44.5GB, 1.02GB/s New Data Upload : 81%|████████ | 4.22GB / 5.20GB, 200MB/s  ...nts/optim_080000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank7.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_080000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_080000_rank4.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_080000_rank5.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_080000_rank6.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_080000_rank1.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_080000_rank2.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 32%|███▏ | 740MB / 2.35GB  Processing Files (9 / 18) : 95%|█████████▍| 42.1GB / 44.5GB, 1.01GB/s New Data Upload : 81%|████████ | 4.26GB / 5.26GB, 201MB/s  ...nts/optim_080000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_080000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_080000_rank3.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_080000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_080000_rank5.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_080000_rank6.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_080000_rank1.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_080000_rank2.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 32%|███▏ | 760MB / 2.35GB  Processing Files (9 / 18) : 95%|█████████▍| 42.2GB / 44.5GB, 984MB/s New Data Upload : 80%|████████ | 4.29GB / 5.33GB, 199MB/s  ...nts/optim_080000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_080000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank3.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_080000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_080000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_080000_rank1.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_080000_rank2.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 33%|███▎ | 783MB / 2.35GB  Processing Files (9 / 18) : 95%|█████████▍| 42.2GB / 44.5GB, 953MB/s New Data Upload : 80%|███████▉ | 4.32GB / 5.40GB, 198MB/s  ...nts/optim_080000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_080000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank3.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_080000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank2.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 34%|███▍ | 803MB / 2.35GB  Processing Files (9 / 18) : 95%|█████████▌| 42.3GB / 44.5GB, 923MB/s New Data Upload : 79%|███████▉ | 4.34GB / 5.47GB, 197MB/s  ...nts/optim_080000_rank0.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_080000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank3.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_080000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_080000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank2.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 35%|███▌ | 829MB / 2.35GB  Processing Files (9 / 18) : 95%|█████████▌| 42.3GB / 44.5GB, 896MB/s New Data Upload : 80%|████████ | 4.37GB / 5.47GB, 197MB/s  ...nts/optim_080000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_080000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_080000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_080000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 37%|███▋ | 858MB / 2.35GB  Processing Files (9 / 18) : 95%|█████████▌| 42.4GB / 44.5GB, 863MB/s New Data Upload : 81%|████████ | 4.41GB / 5.47GB, 195MB/s  ...nts/optim_080000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_080000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_080000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_080000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_080000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 38%|███▊ | 882MB / 2.35GB  Processing Files (9 / 18) : 96%|█████████▌| 42.5GB / 44.5GB, 826MB/s New Data Upload : 80%|████████ | 4.44GB / 5.53GB, 194MB/s  ...nts/optim_080000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_080000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_080000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_080000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 39%|███▉ | 911MB / 2.35GB  Processing Files (9 / 18) : 96%|█████████▌| 42.5GB / 44.5GB, 794MB/s New Data Upload : 80%|███████▉ | 4.47GB / 5.60GB, 193MB/s  ...nts/optim_080000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_080000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank1.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_080000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 40%|████ | 944MB / 2.35GB  Processing Files (9 / 18) : 96%|█████████▌| 42.6GB / 44.5GB, 760MB/s New Data Upload : 80%|███████▉ | 4.51GB / 5.67GB, 192MB/s  ...nts/optim_080000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_080000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 41%|████▏ | 972MB / 2.35GB  Processing Files (9 / 18) : 96%|█████████▌| 42.7GB / 44.5GB, 725MB/s New Data Upload : 80%|████████ | 4.55GB / 5.67GB, 191MB/s  ...nts/optim_080000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 43%|████▎ | 1.01GB / 2.35GB  Processing Files (9 / 18) : 96%|█████████▌| 42.7GB / 44.5GB, 693MB/s New Data Upload : 80%|████████ | 4.59GB / 5.73GB, 190MB/s  ...nts/optim_080000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 44%|████▍ | 1.04GB / 2.35GB  Processing Files (9 / 18) : 96%|█████████▌| 42.8GB / 44.5GB, 656MB/s New Data Upload : 81%|████████ | 4.62GB / 5.73GB, 189MB/s  ...nts/optim_080000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_080000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 46%|████▌ | 1.07GB / 2.35GB  Processing Files (9 / 18) : 96%|█████████▋| 42.8GB / 44.5GB, 618MB/s New Data Upload : 80%|████████ | 4.66GB / 5.80GB, 187MB/s  ...nts/optim_080000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank4.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 47%|████▋ | 1.11GB / 2.35GB  Processing Files (9 / 18) : 96%|█████████▋| 42.9GB / 44.5GB, 579MB/s New Data Upload : 81%|████████ | 4.70GB / 5.80GB, 186MB/s  ...nts/optim_080000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank7.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank5.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...nts/optim_080000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 49%|████▊ | 1.14GB / 2.35GB  Processing Files (9 / 18) : 97%|█████████▋| 42.9GB / 44.5GB, 553MB/s New Data Upload : 81%|████████ | 4.74GB / 5.87GB, 186MB/s  ...nts/optim_080000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 50%|█████ | 1.18GB / 2.35GB  Processing Files (9 / 18) : 97%|█████████▋| 43.0GB / 44.5GB, 522MB/s New Data Upload : 81%|████████ | 4.78GB / 5.93GB, 186MB/s  ...nts/optim_080000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank6.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_080000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 52%|█████▏ | 1.21GB / 2.35GB  Processing Files (9 / 18) : 97%|█████████▋| 43.0GB / 44.5GB, 485MB/s New Data Upload : 81%|████████ | 4.82GB / 5.93GB, 183MB/s  ...nts/optim_080000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank1.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 54%|█████▎ | 1.26GB / 2.35GB  Processing Files (9 / 18) : 97%|█████████▋| 43.1GB / 44.5GB, 461MB/s New Data Upload : 81%|████████ | 4.86GB / 6.00GB, 185MB/s  ...nts/optim_080000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 55%|█████▌ | 1.29GB / 2.35GB  Processing Files (9 / 18) : 97%|█████████▋| 43.2GB / 44.5GB, 436MB/s New Data Upload : 82%|████████▏ | 4.90GB / 6.01GB, 184MB/s  ...nts/optim_080000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 56%|█████▋ | 1.33GB / 2.35GB  Processing Files (9 / 18) : 97%|█████████▋| 43.2GB / 44.5GB, 424MB/s New Data Upload : 82%|████████▏ | 4.94GB / 6.01GB, 183MB/s  ...nts/optim_080000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 58%|█████▊ | 1.37GB / 2.35GB  Processing Files (9 / 18) : 97%|█████████▋| 43.3GB / 44.5GB, 411MB/s New Data Upload : 83%|████████▎ | 4.98GB / 6.01GB, 183MB/s  ...nts/optim_080000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 59%|█████▉ | 1.39GB / 2.35GB  Processing Files (9 / 18) : 97%|█████████▋| 43.3GB / 44.5GB, 401MB/s New Data Upload : 83%|████████▎ | 5.01GB / 6.01GB, 182MB/s  ...nts/optim_080000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_080000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 61%|██████ | 1.43GB / 2.35GB  Processing Files (9 / 18) : 97%|█████████▋| 43.3GB / 44.5GB, 391MB/s New Data Upload : 84%|████████▍ | 5.05GB / 6.01GB, 183MB/s  ...nts/optim_080000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 62%|██████▏ | 1.46GB / 2.35GB  Processing Files (9 / 18) : 98%|█████████▊| 43.4GB / 44.5GB, 378MB/s New Data Upload : 85%|████████▍ | 5.08GB / 6.01GB, 182MB/s  ...nts/optim_080000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 63%|██████▎ | 1.48GB / 2.35GB  Processing Files (9 / 18) : 98%|█████████▊| 43.4GB / 44.5GB, 367MB/s New Data Upload : 85%|████████▌ | 5.11GB / 6.01GB, 181MB/s  ...nts/optim_080000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 64%|██████▍ | 1.51GB / 2.35GB  Processing Files (9 / 18) : 98%|█████████▊| 43.5GB / 44.5GB, 361MB/s New Data Upload : 86%|████████▌ | 5.14GB / 6.01GB, 181MB/s  ...nts/optim_080000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 65%|██████▌ | 1.54GB / 2.35GB  Processing Files (9 / 18) : 98%|█████████▊| 43.5GB / 44.5GB, 349MB/s New Data Upload : 86%|████████▌ | 5.17GB / 6.01GB, 180MB/s  ...nts/optim_080000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_080000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 67%|██████▋ | 1.57GB / 2.35GB  Processing Files (9 / 18) : 98%|█████████▊| 43.6GB / 44.5GB, 340MB/s New Data Upload : 87%|████████▋ | 5.21GB / 6.01GB, 180MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 68%|██████▊ | 1.59GB / 2.35GB  Processing Files (9 / 18) : 98%|█████████▊| 43.6GB / 44.5GB, 329MB/s New Data Upload : 87%|████████▋ | 5.23GB / 6.01GB, 178MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 69%|██████▉ | 1.63GB / 2.35GB  Processing Files (9 / 18) : 98%|█████████▊| 43.6GB / 44.5GB, 323MB/s New Data Upload : 88%|████████▊ | 5.27GB / 6.01GB, 178MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 70%|███████ | 1.65GB / 2.35GB  Processing Files (9 / 18) : 98%|█████████▊| 43.7GB / 44.5GB, 315MB/s New Data Upload : 88%|████████▊ | 5.30GB / 6.01GB, 177MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 72%|███████▏ | 1.68GB / 2.35GB  Processing Files (9 / 18) : 98%|█████████▊| 43.7GB / 44.5GB, 309MB/s New Data Upload : 89%|████████▊ | 5.33GB / 6.01GB, 176MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 73%|███████▎ | 1.71GB / 2.35GB  Processing Files (9 / 18) : 98%|█████████▊| 43.8GB / 44.5GB, 302MB/s New Data Upload : 89%|████████▉ | 5.36GB / 6.01GB, 176MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 74%|███████▎ | 1.73GB / 2.35GB  Processing Files (9 / 18) : 98%|█████████▊| 43.8GB / 44.5GB, 295MB/s New Data Upload : 90%|████████▉ | 5.38GB / 6.01GB, 175MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 75%|███████▍ | 1.76GB / 2.35GB  Processing Files (9 / 18) : 99%|█████████▊| 43.8GB / 44.5GB, 291MB/s New Data Upload : 90%|████████▉ | 5.41GB / 6.01GB, 174MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 76%|███████▌ | 1.78GB / 2.35GB  Processing Files (9 / 18) : 99%|█████████▊| 43.9GB / 44.5GB, 286MB/s New Data Upload : 90%|█████████ | 5.43GB / 6.01GB, 173MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 77%|███████▋ | 1.81GB / 2.35GB  Processing Files (9 / 18) : 99%|█████████▊| 43.9GB / 44.5GB, 278MB/s New Data Upload : 91%|█████████ | 5.46GB / 6.01GB, 172MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 78%|███████▊ | 1.83GB / 2.35GB  Processing Files (9 / 18) : 99%|█████████▉| 43.9GB / 44.5GB, 274MB/s New Data Upload : 91%|█████████▏| 5.48GB / 6.01GB, 172MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 79%|███████▉ | 1.85GB / 2.35GB  Processing Files (9 / 18) : 99%|█████████▉| 43.9GB / 44.5GB, 269MB/s New Data Upload : 92%|█████████▏| 5.51GB / 6.01GB, 170MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 80%|███████▉ | 1.88GB / 2.35GB  Processing Files (9 / 18) : 99%|█████████▉| 44.0GB / 44.5GB, 262MB/s New Data Upload : 92%|█████████▏| 5.53GB / 6.01GB, 168MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 81%|████████ | 1.90GB / 2.35GB  Processing Files (9 / 18) : 99%|█████████▉| 44.0GB / 44.5GB, 257MB/s New Data Upload : 93%|█████████▎| 5.56GB / 6.01GB, 167MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 82%|████████▏ | 1.92GB / 2.35GB  Processing Files (9 / 18) : 99%|█████████▉| 44.0GB / 44.5GB, 251MB/s New Data Upload : 93%|█████████▎| 5.58GB / 6.01GB, 164MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 83%|████████▎ | 1.94GB / 2.35GB  Processing Files (9 / 18) : 99%|█████████▉| 44.1GB / 44.5GB, 248MB/s New Data Upload : 93%|█████████▎| 5.60GB / 6.01GB, 163MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 84%|████████▎ | 1.96GB / 2.35GB  Processing Files (9 / 18) : 99%|█████████▉| 44.1GB / 44.5GB, 243MB/s New Data Upload : 94%|█████████▎| 5.62GB / 6.01GB, 162MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 84%|████████▍ | 1.98GB / 2.35GB  Processing Files (9 / 18) : 99%|█████████▉| 44.1GB / 44.5GB, 236MB/s New Data Upload : 94%|█████████▍| 5.64GB / 6.01GB, 159MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 85%|████████▌ | 2.00GB / 2.35GB  Processing Files (9 / 18) : 99%|█████████▉| 44.1GB / 44.5GB, 232MB/s New Data Upload : 94%|█████████▍| 5.66GB / 6.01GB, 158MB/s  ...nts/optim_080000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 86%|████████▌ | 2.02GB / 2.35GB  Processing Files (9 / 18) : 99%|█████████▉| 44.1GB / 44.5GB, 226MB/s New Data Upload : 95%|█████████▍| 5.68GB / 6.01GB, 156MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 87%|████████▋ | 2.04GB / 2.35GB  Processing Files (17 / 18) : 99%|█████████▉| 44.2GB / 44.5GB, 223MB/s New Data Upload : 95%|█████████▍| 5.70GB / 6.01GB, 155MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 88%|████████▊ | 2.06GB / 2.35GB  Processing Files (17 / 18) : 99%|█████████▉| 44.2GB / 44.5GB, 220MB/s New Data Upload : 95%|█████████▌| 5.72GB / 6.01GB, 153MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 89%|████████▊ | 2.08GB / 2.35GB  Processing Files (17 / 18) : 99%|█████████▉| 44.2GB / 44.5GB, 215MB/s New Data Upload : 96%|█████████▌| 5.74GB / 6.01GB, 152MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 89%|████████▉ | 2.10GB / 2.35GB  Processing Files (17 / 18) : 99%|█████████▉| 44.2GB / 44.5GB, 211MB/s New Data Upload : 96%|█████████▌| 5.75GB / 6.01GB, 150MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 90%|████████▉ | 2.11GB / 2.35GB  Processing Files (17 / 18) : 99%|█████████▉| 44.2GB / 44.5GB, 205MB/s New Data Upload : 96%|█████████▌| 5.77GB / 6.01GB, 148MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 90%|█████████ | 2.12GB / 2.35GB  Processing Files (17 / 18) : 99%|█████████▉| 44.2GB / 44.5GB, 201MB/s New Data Upload : 96%|█████████▋| 5.78GB / 6.01GB, 147MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 91%|█████████ | 2.14GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 198MB/s New Data Upload : 96%|█████████▋| 5.80GB / 6.01GB, 145MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 92%|█████████▏| 2.15GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 193MB/s New Data Upload : 97%|█████████▋| 5.81GB / 6.01GB, 144MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 92%|█████████▏| 2.16GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 189MB/s New Data Upload : 97%|█████████▋| 5.82GB / 6.01GB, 142MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 93%|█████████▎| 2.17GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 184MB/s New Data Upload : 97%|█████████▋| 5.83GB / 6.01GB, 140MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 93%|█████████▎| 2.18GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 179MB/s New Data Upload : 97%|█████████▋| 5.84GB / 6.01GB, 137MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 93%|█████████▎| 2.19GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 174MB/s New Data Upload : 97%|█████████▋| 5.85GB / 6.01GB, 135MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 94%|█████████▎| 2.20GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 168MB/s New Data Upload : 97%|█████████▋| 5.86GB / 6.01GB, 132MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 94%|█████████▎| 2.20GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 162MB/s New Data Upload : 98%|█████████▊| 5.86GB / 6.01GB, 129MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 94%|█████████▍| 2.21GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 156MB/s New Data Upload : 98%|█████████▊| 5.87GB / 6.01GB, 125MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 94%|█████████▍| 2.21GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 152MB/s New Data Upload : 98%|█████████▊| 5.87GB / 6.01GB, 123MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 94%|█████████▍| 2.22GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 148MB/s New Data Upload : 98%|█████████▊| 5.88GB / 6.01GB, 119MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 95%|█████████▍| 2.23GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 143MB/s New Data Upload : 98%|█████████▊| 5.88GB / 6.01GB, 116MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 95%|█████████▌| 2.23GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 138MB/s New Data Upload : 98%|█████████▊| 5.89GB / 6.01GB, 113MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 95%|█████████▌| 2.24GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 133MB/s New Data Upload : 98%|█████████▊| 5.90GB / 6.01GB, 109MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 95%|█████████▌| 2.24GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 129MB/s New Data Upload : 98%|█████████▊| 5.90GB / 6.01GB, 106MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 96%|█████████▌| 2.25GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 124MB/s New Data Upload : 98%|█████████▊| 5.91GB / 6.01GB, 102MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 96%|█████████▌| 2.26GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 119MB/s New Data Upload : 98%|█████████▊| 5.92GB / 6.01GB, 99.2MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 96%|█████████▋| 2.26GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 116MB/s New Data Upload : 99%|█████████▊| 5.92GB / 6.01GB, 96.4MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 97%|█████████▋| 2.27GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 111MB/s New Data Upload : 99%|█████████▊| 5.93GB / 6.01GB, 92.6MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 97%|█████████▋| 2.28GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 108MB/s New Data Upload : 99%|█████████▉| 5.93GB / 6.01GB, 90.5MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 97%|█████████▋| 2.28GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 103MB/s New Data Upload : 99%|█████████▉| 5.94GB / 6.01GB, 87.3MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 97%|█████████▋| 2.29GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 100MB/s New Data Upload : 99%|█████████▉| 5.95GB / 6.01GB, 84.8MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 98%|█████████▊| 2.29GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 95.6MB/s New Data Upload : 99%|█████████▉| 5.95GB / 6.01GB, 82.3MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 98%|█████████▊| 2.30GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 91.8MB/s New Data Upload : 99%|█████████▉| 5.96GB / 6.01GB, 79.8MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 98%|█████████▊| 2.30GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 88.6MB/s New Data Upload : 99%|█████████▉| 5.96GB / 6.01GB, 77.2MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 98%|█████████▊| 2.30GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 84.5MB/s New Data Upload : 99%|█████████▉| 5.96GB / 6.01GB, 74.2MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 98%|█████████▊| 2.31GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 81.6MB/s New Data Upload : 99%|█████████▉| 5.97GB / 6.01GB, 72.3MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 98%|█████████▊| 2.31GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 77.4MB/s New Data Upload : 99%|█████████▉| 5.97GB / 6.01GB, 68.6MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 98%|█████████▊| 2.31GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 69.3MB/s New Data Upload : 99%|█████████▉| 5.97GB / 6.01GB, 63.0MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 98%|█████████▊| 2.31GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 65.9MB/s New Data Upload : 99%|█████████▉| 5.97GB / 6.01GB, 60.2MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 98%|█████████▊| 2.31GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 63.2MB/s New Data Upload : 99%|█████████▉| 5.97GB / 6.01GB, 58.1MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 98%|█████████▊| 2.31GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 59.7MB/s New Data Upload : 99%|█████████▉| 5.97GB / 6.01GB, 55.4MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 98%|█████████▊| 2.31GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 56.0MB/s New Data Upload : 99%|█████████▉| 5.97GB / 6.01GB, 53.1MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 98%|█████████▊| 2.31GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 52.8MB/s New Data Upload : 99%|█████████▉| 5.97GB / 6.01GB, 50.4MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▊| 2.32GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 47.7MB/s New Data Upload : 99%|█████████▉| 5.97GB / 6.01GB, 45.9MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▊| 2.32GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 41.6MB/s New Data Upload : 99%|█████████▉| 5.98GB / 6.01GB, 41.0MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▊| 2.32GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 37.4MB/s New Data Upload : 99%|█████████▉| 5.98GB / 6.01GB, 37.1MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▊| 2.32GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 33.7MB/s New Data Upload : 100%|█████████▉| 5.98GB / 6.01GB, 33.6MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▉| 2.32GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 31.5MB/s New Data Upload : 100%|█████████▉| 5.98GB / 6.01GB, 31.4MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▉| 2.32GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 27.7MB/s New Data Upload : 100%|█████████▉| 5.98GB / 6.01GB, 27.7MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▉| 2.32GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 26.1MB/s New Data Upload : 100%|█████████▉| 5.98GB / 6.01GB, 26.1MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▉| 2.33GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 22.6MB/s New Data Upload : 100%|█████████▉| 5.99GB / 6.01GB, 22.6MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▉| 2.33GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 21.3MB/s New Data Upload : 100%|█████████▉| 5.99GB / 6.01GB, 21.3MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▉| 2.33GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 20.1MB/s New Data Upload : 100%|█████████▉| 5.99GB / 6.01GB, 20.1MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▉| 2.33GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 18.9MB/s New Data Upload : 100%|█████████▉| 5.99GB / 6.01GB, 18.9MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▉| 2.33GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.5GB / 44.5GB, 17.9MB/s New Data Upload : 100%|█████████▉| 5.99GB / 6.01GB, 17.9MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▉| 2.33GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.5GB / 44.5GB, 16.7MB/s New Data Upload : 100%|█████████▉| 5.99GB / 6.01GB, 16.7MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▉| 2.34GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.5GB / 44.5GB, 15.8MB/s New Data Upload : 100%|█████████▉| 5.99GB / 6.01GB, 15.8MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 99%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|█████████▉| 2.34GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.5GB / 44.5GB, 14.5MB/s New Data Upload : 100%|█████████▉| 6.00GB / 6.01GB, 14.5MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|█████████▉| 2.34GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.5GB / 44.5GB, 14.0MB/s New Data Upload : 100%|█████████▉| 6.00GB / 6.01GB, 14.0MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|█████████▉| 2.34GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.5GB / 44.5GB, 13.9MB/s New Data Upload : 100%|█████████▉| 6.00GB / 6.01GB, 13.9MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|█████████▉| 2.34GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.5GB / 44.5GB, 13.3MB/s New Data Upload : 100%|█████████▉| 6.00GB / 6.01GB, 13.3MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|█████████▉| 2.35GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.5GB / 44.5GB, 12.8MB/s New Data Upload : 100%|█████████▉| 6.00GB / 6.01GB, 12.8MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|█████████▉| 2.35GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.5GB / 44.5GB, 12.7MB/s New Data Upload : 100%|█████████▉| 6.01GB / 6.01GB, 12.7MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|█████████▉| 2.35GB / 2.35GB  Processing Files (17 / 18) : 100%|█████████▉| 44.5GB / 44.5GB, 12.1MB/s New Data Upload : 100%|█████████▉| 6.01GB / 6.01GB, 12.1MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|██████████| 2.35GB / 2.35GB  Processing Files (18 / 18) : 100%|██████████| 44.5GB / 44.5GB, 9.03MB/s New Data Upload : 100%|██████████| 6.01GB / 6.01GB, 9.03MB/s  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_080000.pt: 100%|██████████| 2.35GB / 2.35GB  Processing Files (18 / 18) : 100%|██████████| 44.5GB / 44.5GB, 4.19MB/s New Data Upload : 100%|██████████| 6.01GB / 6.01GB, 4.19MB/s ...nts/optim_080000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_080000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_080000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_080000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_080000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_080000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_080000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_080000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB ...eckpoints/model_080000.pt: 100%|██████████| 2.35GB / 2.35GB [22:01:09] Step 80000 uploaded [23:01:09] Uploading step 85000... Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...nts/optim_075000_rank5.pt: 5%|▍ | 120MB / 2.49GB  ...nts/optim_075000_rank7.pt: 5%|▍ | 120MB / 2.49GB  ...eckpoints/model_075000.pt: 5%|▍ | 112MB / 2.35GB  ...nts/optim_075000_rank0.pt: 5%|▍ | 112MB / 2.49GB  ...nts/optim_075000_rank5.pt: 5%|▍ | 120MB / 2.49GB  ...nts/optim_075000_rank7.pt: 5%|▍ | 120MB / 2.49GB  ...eckpoints/model_075000.pt: 5%|▍ | 112MB / 2.35GB  ...nts/optim_075000_rank0.pt: 5%|▍ | 112MB / 2.49GB  Processing Files (0 / 4) : 1%| | 464MB / 44.5GB, ???B/s ...nts/optim_075000_rank5.pt: 10%|▉ | 248MB / 2.49GB  ...nts/optim_075000_rank7.pt: 10%|▉ | 248MB / 2.49GB  ...eckpoints/model_075000.pt: 10%|▉ | 232MB / 2.35GB  ...nts/optim_075000_rank0.pt: 10%|▉ | 240MB / 2.49GB  Processing Files (0 / 4) : 2%|▏ | 968MB / 44.5GB, 2.52GB/s ...nts/optim_075000_rank5.pt: 14%|█▍ | 360MB / 2.49GB  ...nts/optim_075000_rank7.pt: 15%|█▌ | 376MB / 2.49GB  ...eckpoints/model_075000.pt: 15%|█▍ | 352MB / 2.35GB  ...nts/optim_075000_rank0.pt: 14%|█▍ | 352MB / 2.49GB  Processing Files (0 / 4) : 3%|▎ | 1.44GB / 44.5GB, 2.44GB/s ...nts/optim_075000_rank5.pt: 20%|█▉ | 488MB / 2.49GB  ...nts/optim_075000_rank7.pt: 21%|██ | 520MB / 2.49GB  ...eckpoints/model_075000.pt: 21%|██ | 488MB / 2.35GB  ...nts/optim_075000_rank0.pt: 19%|█▉ | 480MB / 2.49GB  Processing Files (0 / 4) : 4%|▍ | 1.98GB / 44.5GB, 2.52GB/s ...eckpoints/model_085000.pt: 0%| | 5.40MB / 2.35GB  ...nts/optim_075000_rank5.pt: 25%|██▌ | 632MB / 2.49GB  ...nts/optim_075000_rank7.pt: 27%|██▋ | 672MB / 2.49GB  ...eckpoints/model_075000.pt: 27%|██▋ | 640MB / 2.35GB  ...nts/optim_075000_rank0.pt: 25%|██▍ | 616MB / 2.49GB  ...eckpoints/model_085000.pt: 0%| | 5.40MB / 2.35GB  Processing Files (0 / 5) : 6%|▌ | 2.57GB / 44.5GB, 2.63GB/s New Data Upload : 1%| | 2.32MB / 335MB, 2.90MB/s  ...nts/optim_075000_rank5.pt: 31%|███ | 768MB / 2.49GB  ...nts/optim_075000_rank7.pt: 33%|███▎ | 808MB / 2.49GB  ...eckpoints/model_075000.pt: 33%|███▎ | 768MB / 2.35GB  ...nts/optim_075000_rank0.pt: 30%|███ | 752MB / 2.49GB  ...eckpoints/model_085000.pt: 0%| | 5.40MB / 2.35GB  Processing Files (0 / 5) : 7%|▋ | 3.10GB / 44.5GB, 2.64GB/s ...nts/optim_075000_rank5.pt: 35%|███▌ | 880MB / 2.49GB  ...nts/optim_075000_rank7.pt: 38%|███▊ | 936MB / 2.49GB  ...eckpoints/model_075000.pt: 38%|███▊ | 888MB / 2.35GB  ...nts/optim_075000_rank0.pt: 35%|███▌ | 872MB / 2.49GB  ...eckpoints/model_085000.pt: 0%| | 5.40MB / 2.35GB  Processing Files (0 / 5) : 8%|▊ | 3.58GB / 44.5GB, 2.60GB/s ...nts/optim_075000_rank5.pt: 40%|████ | 1.00GB / 2.49GB  ...nts/optim_075000_rank7.pt: 42%|████▏ | 1.06GB / 2.49GB  ...eckpoints/model_075000.pt: 43%|████▎ | 1.01GB / 2.35GB  ...nts/optim_075000_rank0.pt: 40%|███▉ | 992MB / 2.49GB  ...eckpoints/model_085000.pt: 0%| | 7.14MB / 2.35GB  Processing Files (0 / 5) : 9%|▉ | 4.06GB / 44.5GB, 2.57GB/s New Data Upload : 1%| | 4.06MB / 335MB, 2.90MB/s  ...nts/optim_075000_rank5.pt: 45%|████▌ | 1.13GB / 2.49GB  ...nts/optim_075000_rank7.pt: 48%|████▊ | 1.19GB / 2.49GB  ...eckpoints/model_075000.pt: 48%|████▊ | 1.13GB / 2.35GB  ...nts/optim_075000_rank0.pt: 45%|████▍ | 1.11GB / 2.49GB  ...eckpoints/model_085000.pt: 0%| | 11.2MB / 2.35GB  Processing Files (0 / 5) : 10%|█ | 4.57GB / 44.5GB, 2.57GB/s New Data Upload : 2%|▏ | 8.12MB / 335MB, 5.08MB/s  ...nts/optim_075000_rank5.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_075000_rank7.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...eckpoints/model_075000.pt: 54%|█████▍ | 1.26GB / 2.35GB  ...nts/optim_075000_rank0.pt: 50%|████▉ | 1.23GB / 2.49GB  ...eckpoints/model_085000.pt: 1%| | 19.3MB / 2.35GB  Processing Files (0 / 5) : 11%|█▏ | 5.08GB / 44.5GB, 2.57GB/s New Data Upload : 5%|▍ | 16.2MB / 335MB, 9.03MB/s  ...nts/optim_075000_rank5.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_075000_rank7.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...eckpoints/model_075000.pt: 59%|█████▊ | 1.38GB / 2.35GB  ...nts/optim_075000_rank0.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...eckpoints/model_085000.pt: 1%| | 23.4MB / 2.35GB  Processing Files (0 / 5) : 13%|█▎ | 5.57GB / 44.5GB, 2.55GB/s New Data Upload : 6%|▌ | 20.3MB / 335MB, 10.2MB/s  ...nts/optim_075000_rank5.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_075000_rank7.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...eckpoints/model_075000.pt: 64%|██████▎ | 1.50GB / 2.35GB  ...nts/optim_075000_rank0.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...eckpoints/model_085000.pt: 1%| | 28.0MB / 2.35GB  Processing Files (0 / 5) : 14%|█▎ | 6.06GB / 44.5GB, 2.54GB/s New Data Upload : 7%|▋ | 24.9MB / 335MB, 11.3MB/s  ...nts/optim_075000_rank5.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_075000_rank7.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...eckpoints/model_075000.pt: 69%|██████▉ | 1.62GB / 2.35GB  ...nts/optim_075000_rank0.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...eckpoints/model_085000.pt: 1%|▏ | 33.2MB / 2.35GB  Processing Files (0 / 5) : 15%|█▍ | 6.56GB / 44.5GB, 2.54GB/s New Data Upload : 9%|▉ | 30.2MB / 335MB, 12.6MB/s  ...nts/optim_075000_rank5.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_075000_rank7.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...eckpoints/model_075000.pt: 75%|███████▍ | 1.75GB / 2.35GB  ...nts/optim_075000_rank0.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...eckpoints/model_085000.pt: 2%|▏ | 37.9MB / 2.35GB  Processing Files (0 / 5) : 16%|█▌ | 7.07GB / 44.5GB, 2.54GB/s New Data Upload : 10%|█ | 34.8MB / 335MB, 13.4MB/s  ...nts/optim_075000_rank5.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_075000_rank7.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...eckpoints/model_075000.pt: 80%|████████ | 1.88GB / 2.35GB  ...nts/optim_075000_rank0.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...eckpoints/model_085000.pt: 2%|▏ | 44.9MB / 2.35GB  Processing Files (0 / 5) : 17%|█▋ | 7.58GB / 44.5GB, 2.54GB/s New Data Upload : 12%|█▏ | 41.8MB / 335MB, 14.9MB/s  ...nts/optim_075000_rank5.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_075000_rank7.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...eckpoints/model_075000.pt: 85%|████████▌ | 2.00GB / 2.35GB  ...nts/optim_075000_rank0.pt: 79%|███████▉ | 1.98GB / 2.49GB  ...eckpoints/model_085000.pt: 2%|▏ | 48.3MB / 2.35GB  Processing Files (0 / 5) : 18%|█▊ | 8.07GB / 44.5GB, 2.54GB/s New Data Upload : 13%|█▎ | 45.3MB / 335MB, 15.1MB/s  ...nts/optim_075000_rank5.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_075000_rank7.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...eckpoints/model_075000.pt: 91%|█████████ | 2.13GB / 2.35GB  ...nts/optim_075000_rank0.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...eckpoints/model_085000.pt: 2%|▏ | 55.9MB / 2.35GB  Processing Files (0 / 5) : 19%|█▉ | 8.58GB / 44.5GB, 2.54GB/s New Data Upload : 16%|█▌ | 52.8MB / 335MB, 16.5MB/s  ...nts/optim_075000_rank5.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_075000_rank7.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...eckpoints/model_075000.pt: 96%|█████████▌| 2.26GB / 2.35GB  ...nts/optim_075000_rank0.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...eckpoints/model_085000.pt: 2%|▏ | 55.9MB / 2.35GB  Processing Files (0 / 5) : 20%|██ | 9.07GB / 44.5GB, 2.53GB/s ...nts/optim_075000_rank5.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_075000_rank7.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 60.5MB / 2.35GB  Processing Files (1 / 5) : 21%|██▏ | 9.53GB / 44.5GB, 2.52GB/s New Data Upload : 17%|█▋ | 57.4MB / 335MB, 16.0MB/s  ...nts/optim_075000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 65.2MB / 2.35GB  Processing Files (2 / 5) : 22%|██▏ | 9.86GB / 44.5GB, 2.47GB/s New Data Upload : 19%|█▊ | 62.1MB / 335MB, 16.3MB/s  ...nts/optim_075000_rank2.pt: 3%|▎ | 80.0MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 80.0MB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 68.6MB / 2.35GB  ...nts/optim_075000_rank2.pt: 3%|▎ | 80.0MB / 2.49GB  ...nts/optim_075000_rank1.pt: 3%|▎ | 80.0MB / 2.49GB  Processing Files (4 / 7) : 23%|██▎ | 10.0GB / 44.5GB, 2.39GB/s New Data Upload : 16%|█▋ | 65.6MB / 403MB, 16.4MB/s  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 69.8MB / 2.35GB  ...nts/optim_075000_rank2.pt: 8%|▊ | 192MB / 2.49GB  ...nts/optim_075000_rank1.pt: 8%|▊ | 208MB / 2.49GB  Processing Files (4 / 7) : 23%|██▎ | 10.3GB / 44.5GB, 2.34GB/s New Data Upload : 14%|█▍ | 66.7MB / 470MB, 15.9MB/s  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 69.8MB / 2.35GB  ...nts/optim_075000_rank2.pt: 13%|█▎ | 320MB / 2.49GB  ...nts/optim_075000_rank1.pt: 14%|█▎ | 336MB / 2.49GB  Processing Files (4 / 7) : 24%|██▎ | 10.5GB / 44.5GB, 2.29GB/s ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 69.8MB / 2.35GB  ...nts/optim_075000_rank2.pt: 18%|█▊ | 440MB / 2.49GB  ...nts/optim_075000_rank1.pt: 18%|█▊ | 456MB / 2.49GB  Processing Files (4 / 7) : 24%|██▍ | 10.8GB / 44.5GB, 2.24GB/s ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 69.8MB / 2.35GB  ...nts/optim_075000_rank2.pt: 23%|██▎ | 560MB / 2.49GB  ...nts/optim_075000_rank1.pt: 23%|██▎ | 576MB / 2.49GB  Processing Files (4 / 7) : 25%|██▍ | 11.0GB / 44.5GB, 2.20GB/s ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_075000_rank2.pt: 28%|██▊ | 688MB / 2.49GB  ...nts/optim_075000_rank1.pt: 28%|██▊ | 688MB / 2.49GB  Processing Files (4 / 7) : 25%|██▌ | 11.3GB / 44.5GB, 2.16GB/s New Data Upload : 14%|█▍ | 67.1MB / 470MB, 13.4MB/s  ...nts/optim_085000_rank4.pt: 0%| | 4.11MB / 2.49GB  ...nts/optim_085000_rank1.pt: 0%| | 27.4kB / 2.49GB  ...nts/optim_085000_rank3.pt: 0%| | 27.4kB / 2.49GB  ...nts/optim_075000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_075000_rank2.pt: 33%|███▎ | 816MB / 2.49GB  ...nts/optim_075000_rank1.pt: 33%|███▎ | 816MB / 2.49GB  ...nts/optim_085000_rank4.pt: 0%| | 4.11MB / 2.49GB  ...nts/optim_085000_rank1.pt: 0%| | 27.4kB / 2.49GB  [+ 1 files] : 0%| | 614kB / 4.97GB  Processing Files (4 / 11) : 26%|██▌ | 11.5GB / 44.5GB, 2.12GB/s New Data Upload : 15%|█▌ | 71.7MB / 470MB, 13.8MB/s  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_075000_rank2.pt: 38%|███▊ | 936MB / 2.49GB  ...nts/optim_075000_rank1.pt: 38%|███▊ | 936MB / 2.49GB  ...nts/optim_085000_rank4.pt: 0%| | 6.46MB / 2.49GB  ...nts/optim_085000_rank1.pt: 0%| | 27.4kB / 2.49GB  ...nts/optim_085000_rank3.pt: 0%| | 27.4kB / 2.49GB  ...nts/optim_085000_rank7.pt: 0%| | 587kB / 2.49GB  Processing Files (4 / 11) : 26%|██▋ | 11.8GB / 44.5GB, 2.09GB/s New Data Upload : 12%|█▏ | 74.1MB / 604MB, 13.7MB/s  ...nts/optim_075000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_075000_rank2.pt: 42%|████▏ | 1.06GB / 2.49GB  ...nts/optim_075000_rank1.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_085000_rank4.pt: 0%| | 11.2MB / 2.49GB  ...nts/optim_085000_rank1.pt: 0%| | 110kB / 2.49GB  ...nts/optim_085000_rank3.pt: 0%| | 110kB / 2.49GB  [+ 1 files] : 0%| | 2.46MB / 4.97GB  Processing Files (4 / 12) : 27%|██▋ | 12.0GB / 44.5GB, 2.06GB/s New Data Upload : 13%|█▎ | 80.6MB / 604MB, 14.4MB/s  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_075000_rank2.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_075000_rank1.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_085000_rank4.pt: 1%| | 14.1MB / 2.49GB  ...nts/optim_085000_rank1.pt: 0%| | 833kB / 2.49GB  ...nts/optim_085000_rank3.pt: 0%| | 833kB / 2.49GB  ...nts/optim_085000_rank7.pt: 0%| | 5.28MB / 2.49GB  ...nts/optim_085000_rank2.pt: 0%| | 3.72MB / 2.49GB  Processing Files (4 / 12) : 28%|██▊ | 12.3GB / 44.5GB, 2.04GB/s New Data Upload : 14%|█▍ | 87.0MB / 604MB, 15.0MB/s  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 70.1MB / 2.35GB  ...nts/optim_075000_rank2.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_075000_rank1.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_085000_rank4.pt: 1%| | 18.8MB / 2.49GB  ...nts/optim_085000_rank1.pt: 0%| | 970kB / 2.49GB  ...nts/optim_085000_rank3.pt: 0%| | 970kB / 2.49GB  ...nts/optim_085000_rank7.pt: 0%| | 8.22MB / 2.49GB  ...nts/optim_085000_rank2.pt: 0%| | 3.85MB / 2.49GB  Processing Files (4 / 12) : 28%|██▊ | 12.6GB / 44.5GB, 2.02GB/s New Data Upload : 16%|█▌ | 94.6MB / 604MB, 15.8MB/s  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 72.4MB / 2.35GB  ...nts/optim_075000_rank2.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_075000_rank1.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 1%| | 24.1MB / 2.49GB  ...nts/optim_085000_rank1.pt: 0%| | 3.09MB / 2.49GB  ...nts/optim_085000_rank3.pt: 0%| | 3.09MB / 2.49GB  ...nts/optim_085000_rank7.pt: 1%| | 15.8MB / 2.49GB  ...nts/optim_085000_rank2.pt: 0%| | 5.97MB / 2.49GB  Processing Files (4 / 12) : 29%|██▉ | 12.9GB / 44.5GB, 2.00GB/s New Data Upload : 17%|█▋ | 112MB / 671MB, 18.0MB/s  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 72.4MB / 2.35GB  ...nts/optim_075000_rank2.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_075000_rank1.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_085000_rank4.pt: 1%| | 29.3MB / 2.49GB  ...nts/optim_085000_rank1.pt: 0%| | 7.02MB / 2.49GB  ...nts/optim_085000_rank3.pt: 0%| | 7.02MB / 2.49GB  ...nts/optim_085000_rank7.pt: 1%| | 24.6MB / 2.49GB  ...nts/optim_085000_rank2.pt: 0%| | 9.90MB / 2.49GB  Processing Files (4 / 12) : 30%|██▉ | 13.2GB / 44.5GB, 1.99GB/s New Data Upload : 18%|█▊ | 129MB / 738MB, 20.2MB/s  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 72.4MB / 2.35GB  ...nts/optim_075000_rank2.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_075000_rank1.pt: 71%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_085000_rank4.pt: 1%|▏ | 32.3MB / 2.49GB  ...nts/optim_085000_rank1.pt: 0%| | 9.58MB / 2.49GB  ...nts/optim_085000_rank3.pt: 0%| | 9.58MB / 2.49GB  ...nts/optim_085000_rank7.pt: 1%| | 29.3MB / 2.49GB  ...nts/optim_085000_rank2.pt: 1%| | 12.5MB / 2.49GB  Processing Files (4 / 12) : 30%|███ | 13.5GB / 44.5GB, 1.97GB/s New Data Upload : 19%|█▉ | 139MB / 738MB, 21.1MB/s  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 72.4MB / 2.35GB  ...nts/optim_075000_rank2.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_075000_rank1.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_085000_rank4.pt: 1%|▏ | 36.4MB / 2.49GB  ...nts/optim_085000_rank1.pt: 1%| | 18.2MB / 2.49GB  ...nts/optim_085000_rank3.pt: 1%| | 21.1MB / 2.49GB  ...nts/optim_085000_rank7.pt: 2%|▏ | 37.6MB / 2.49GB  ...nts/optim_085000_rank2.pt: 1%| | 21.0MB / 2.49GB  Processing Files (4 / 12) : 31%|███ | 13.8GB / 44.5GB, 1.96GB/s New Data Upload : 21%|██▏ | 159MB / 738MB, 23.3MB/s  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 76.3MB / 2.35GB  ...nts/optim_075000_rank2.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_075000_rank1.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_085000_rank4.pt: 2%|▏ | 38.2MB / 2.49GB  ...nts/optim_085000_rank1.pt: 1%| | 21.5MB / 2.49GB  ...nts/optim_085000_rank3.pt: 1%| | 24.4MB / 2.49GB  ...nts/optim_085000_rank7.pt: 2%|▏ | 45.8MB / 2.49GB  ...nts/optim_085000_rank2.pt: 1%| | 24.4MB / 2.49GB  Processing Files (4 / 12) : 32%|███▏ | 14.1GB / 44.5GB, 1.94GB/s New Data Upload : 24%|██▍ | 175MB / 738MB, 25.1MB/s  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 3%|▎ | 76.3MB / 2.35GB  ...nts/optim_075000_rank2.pt: 86%|████████▋ | 2.14GB / 2.49GB  ...nts/optim_075000_rank1.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_085000_rank4.pt: 2%|▏ | 42.3MB / 2.49GB  ...nts/optim_085000_rank1.pt: 1%|▏ | 33.5MB / 2.49GB  ...nts/optim_085000_rank3.pt: 1%|▏ | 33.5MB / 2.49GB  ...nts/optim_085000_rank7.pt: 2%|▏ | 52.8MB / 2.49GB  ...nts/optim_085000_rank2.pt: 1%|▏ | 33.4MB / 2.49GB  Processing Files (4 / 12) : 32%|███▏ | 14.4GB / 44.5GB, 1.94GB/s New Data Upload : 26%|██▌ | 191MB / 738MB, 26.5MB/s  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 4%|▎ | 82.5MB / 2.35GB  ...nts/optim_075000_rank2.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_075000_rank1.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_085000_rank4.pt: 2%|▏ | 48.2MB / 2.49GB  ...nts/optim_085000_rank1.pt: 2%|▏ | 40.1MB / 2.49GB  ...nts/optim_085000_rank3.pt: 2%|▏ | 40.1MB / 2.49GB  ...nts/optim_085000_rank7.pt: 2%|▏ | 57.5MB / 2.49GB  ...nts/optim_085000_rank2.pt: 2%|▏ | 40.0MB / 2.49GB  Processing Files (4 / 12) : 33%|███▎ | 14.7GB / 44.5GB, 1.92GB/s New Data Upload : 26%|██▌ | 210MB / 805MB, 28.4MB/s  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 4%|▎ | 87.0MB / 2.35GB  ...nts/optim_075000_rank2.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_075000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank4.pt: 2%|▏ | 56.9MB / 2.49GB  ...nts/optim_085000_rank1.pt: 2%|▏ | 48.2MB / 2.49GB  ...nts/optim_085000_rank3.pt: 2%|▏ | 48.2MB / 2.49GB  ...nts/optim_085000_rank7.pt: 3%|▎ | 68.1MB / 2.49GB  ...nts/optim_085000_rank2.pt: 2%|▏ | 48.0MB / 2.49GB  Processing Files (4 / 12) : 34%|███▍ | 15.0GB / 44.5GB, 1.92GB/s New Data Upload : 29%|██▉ | 232MB / 805MB, 30.5MB/s  ...eckpoints/model_075000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 4%|▍ | 92.0MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 3%|▎ | 65.0MB / 2.49GB  ...nts/optim_085000_rank1.pt: 2%|▏ | 59.0MB / 2.49GB  ...nts/optim_085000_rank3.pt: 2%|▏ | 58.8MB / 2.49GB  ...nts/optim_085000_rank7.pt: 3%|▎ | 80.8MB / 2.49GB  [+ 1 files] : 3%|▎ | 147MB / 4.97GB  Processing Files (6 / 13) : 34%|███▍ | 15.3GB / 44.5GB, 1.90GB/s New Data Upload : 32%|███▏ | 257MB / 805MB, 33.0MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 4%|▍ | 97.0MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 3%|▎ | 77.6MB / 2.49GB  ...nts/optim_085000_rank1.pt: 3%|▎ | 72.3MB / 2.49GB  ...nts/optim_085000_rank3.pt: 3%|▎ | 71.8MB / 2.49GB  ...nts/optim_085000_rank7.pt: 4%|▎ | 91.6MB / 2.49GB  ...nts/optim_085000_rank2.pt: 3%|▎ | 71.8MB / 2.49GB  ...nts/optim_075000_rank6.pt: 9%|▉ | 224MB / 2.49GB  Processing Files (6 / 13) : 35%|███▍ | 15.5GB / 44.5GB, 1.88GB/s New Data Upload : 34%|███▍ | 277MB / 805MB, 34.6MB/s  ...nts/optim_075000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 4%|▍ | 99.3MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 4%|▍ | 95.3MB / 2.49GB  ...nts/optim_085000_rank1.pt: 4%|▎ | 90.1MB / 2.49GB  ...nts/optim_085000_rank3.pt: 4%|▎ | 89.1MB / 2.49GB  ...nts/optim_085000_rank7.pt: 4%|▍ | 108MB / 2.49GB  ...nts/optim_085000_rank2.pt: 4%|▎ | 89.4MB / 2.49GB  [+ 1 files] : 9%|▉ | 437MB / 4.97GB  Processing Files (6 / 14) : 35%|███▌ | 15.8GB / 44.5GB, 1.87GB/s New Data Upload : 35%|███▍ | 301MB / 872MB, 36.8MB/s  ...eckpoints/model_085000.pt: 4%|▍ | 104MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 5%|▍ | 112MB / 2.49GB  ...nts/optim_085000_rank1.pt: 4%|▍ | 105MB / 2.49GB  ...nts/optim_085000_rank3.pt: 4%|▍ | 103MB / 2.49GB  ...nts/optim_085000_rank7.pt: 5%|▍ | 120MB / 2.49GB  ...nts/optim_085000_rank2.pt: 4%|▍ | 104MB / 2.49GB  ...nts/optim_075000_rank6.pt: 20%|█▉ | 488MB / 2.49GB  ...nts/optim_085000_rank5.pt: 4%|▍ | 97.1MB / 2.49GB  Processing Files (6 / 14) : 36%|███▌ | 16.0GB / 44.5GB, 1.85GB/s New Data Upload : 35%|███▍ | 327MB / 939MB, 39.0MB/s  ...eckpoints/model_085000.pt: 5%|▍ | 108MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 5%|▌ | 132MB / 2.49GB  ...nts/optim_085000_rank1.pt: 5%|▍ | 122MB / 2.49GB  ...nts/optim_085000_rank3.pt: 5%|▍ | 119MB / 2.49GB  ...nts/optim_085000_rank7.pt: 5%|▌ | 133MB / 2.49GB  ...nts/optim_085000_rank2.pt: 5%|▍ | 121MB / 2.49GB  ...nts/optim_075000_rank6.pt: 25%|██▍ | 616MB / 2.49GB  ...nts/optim_085000_rank5.pt: 5%|▍ | 116MB / 2.49GB  Processing Files (6 / 14) : 37%|███▋ | 16.2GB / 44.5GB, 1.83GB/s New Data Upload : 35%|███▌ | 354MB / 1.01GB, 41.2MB/s  ...eckpoints/model_085000.pt: 5%|▍ | 114MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 6%|▋ | 158MB / 2.49GB  ...nts/optim_085000_rank1.pt: 6%|▌ | 144MB / 2.49GB  ...nts/optim_085000_rank3.pt: 6%|▌ | 141MB / 2.49GB  ...nts/optim_085000_rank7.pt: 6%|▌ | 152MB / 2.49GB  ...nts/optim_085000_rank2.pt: 6%|▌ | 144MB / 2.49GB  ...nts/optim_075000_rank6.pt: 31%|███ | 768MB / 2.49GB  ...nts/optim_085000_rank5.pt: 6%|▌ | 137MB / 2.49GB  Processing Files (6 / 14) : 37%|███▋ | 16.5GB / 44.5GB, 1.83GB/s New Data Upload : 39%|███▉ | 392MB / 1.01GB, 44.5MB/s  ...eckpoints/model_085000.pt: 5%|▌ | 118MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 7%|▋ | 176MB / 2.49GB  ...nts/optim_085000_rank1.pt: 6%|▋ | 161MB / 2.49GB  ...nts/optim_085000_rank3.pt: 6%|▋ | 157MB / 2.49GB  ...nts/optim_085000_rank7.pt: 7%|▋ | 167MB / 2.49GB  ...nts/optim_085000_rank2.pt: 6%|▋ | 160MB / 2.49GB  ...nts/optim_075000_rank6.pt: 36%|███▋ | 904MB / 2.49GB  ...nts/optim_085000_rank5.pt: 6%|▌ | 153MB / 2.49GB  Processing Files (6 / 14) : 38%|███▊ | 16.8GB / 44.5GB, 1.81GB/s New Data Upload : 36%|███▋ | 416MB / 1.14GB, 46.2MB/s  ...eckpoints/model_085000.pt: 5%|▌ | 120MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 8%|▊ | 198MB / 2.49GB  ...nts/optim_085000_rank1.pt: 7%|▋ | 182MB / 2.49GB  ...nts/optim_085000_rank3.pt: 7%|▋ | 177MB / 2.49GB  ...nts/optim_085000_rank7.pt: 7%|▋ | 186MB / 2.49GB  ...nts/optim_085000_rank2.pt: 7%|▋ | 181MB / 2.49GB  ...nts/optim_075000_rank6.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_085000_rank5.pt: 7%|▋ | 173MB / 2.49GB  Processing Files (6 / 14) : 38%|███▊ | 17.1GB / 44.5GB, 1.80GB/s New Data Upload : 39%|███▉ | 442MB / 1.14GB, 48.1MB/s  ...eckpoints/model_085000.pt: 5%|▌ | 125MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 9%|▊ | 216MB / 2.49GB  ...nts/optim_085000_rank1.pt: 8%|▊ | 198MB / 2.49GB  ...nts/optim_085000_rank3.pt: 8%|▊ | 193MB / 2.49GB  ...nts/optim_085000_rank7.pt: 8%|▊ | 198MB / 2.49GB  ...nts/optim_085000_rank2.pt: 8%|▊ | 197MB / 2.49GB  ...nts/optim_075000_rank6.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_085000_rank5.pt: 8%|▊ | 189MB / 2.49GB  Processing Files (6 / 14) : 39%|███▉ | 17.3GB / 44.5GB, 1.79GB/s New Data Upload : 39%|███▉ | 470MB / 1.21GB, 50.0MB/s  ...eckpoints/model_085000.pt: 6%|▌ | 131MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 9%|▉ | 229MB / 2.49GB  ...nts/optim_085000_rank1.pt: 8%|▊ | 211MB / 2.49GB  ...nts/optim_085000_rank3.pt: 8%|▊ | 206MB / 2.49GB  ...nts/optim_085000_rank7.pt: 8%|▊ | 209MB / 2.49GB  ...nts/optim_085000_rank2.pt: 8%|▊ | 210MB / 2.49GB  ...nts/optim_075000_rank6.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_085000_rank5.pt: 8%|▊ | 201MB / 2.49GB  Processing Files (6 / 14) : 39%|███▉ | 17.5GB / 44.5GB, 1.78GB/s New Data Upload : 41%|████ | 492MB / 1.21GB, 51.3MB/s  ...eckpoints/model_085000.pt: 6%|▌ | 137MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 10%|▉ | 240MB / 2.49GB  ...nts/optim_085000_rank1.pt: 9%|▉ | 222MB / 2.49GB  ...nts/optim_085000_rank3.pt: 9%|▊ | 217MB / 2.49GB  ...nts/optim_085000_rank7.pt: 9%|▉ | 218MB / 2.49GB  ...nts/optim_085000_rank2.pt: 9%|▉ | 221MB / 2.49GB  ...nts/optim_075000_rank6.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 9%|▊ | 212MB / 2.49GB  Processing Files (6 / 14) : 40%|███▉ | 17.7GB / 44.5GB, 1.76GB/s New Data Upload : 42%|████▏ | 511MB / 1.21GB, 52.2MB/s  ...eckpoints/model_085000.pt: 6%|▋ | 147MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 10%|█ | 252MB / 2.49GB  ...nts/optim_085000_rank1.pt: 9%|▉ | 234MB / 2.49GB  ...nts/optim_085000_rank3.pt: 9%|▉ | 227MB / 2.49GB  ...nts/optim_085000_rank7.pt: 9%|▉ | 230MB / 2.49GB  ...nts/optim_085000_rank2.pt: 9%|▉ | 232MB / 2.49GB  ...nts/optim_075000_rank6.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_085000_rank5.pt: 9%|▉ | 222MB / 2.49GB  Processing Files (6 / 14) : 40%|████ | 17.9GB / 44.5GB, 1.75GB/s New Data Upload : 42%|████▏ | 537MB / 1.27GB, 53.7MB/s  ...eckpoints/model_085000.pt: 7%|▋ | 156MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 10%|█ | 261MB / 2.49GB  ...nts/optim_085000_rank1.pt: 10%|▉ | 241MB / 2.49GB  ...nts/optim_085000_rank3.pt: 9%|▉ | 235MB / 2.49GB  ...nts/optim_085000_rank7.pt: 10%|▉ | 237MB / 2.49GB  ...nts/optim_085000_rank2.pt: 10%|▉ | 239MB / 2.49GB  ...nts/optim_075000_rank6.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_085000_rank5.pt: 9%|▉ | 229MB / 2.49GB  Processing Files (6 / 14) : 41%|████ | 18.1GB / 44.5GB, 1.73GB/s New Data Upload : 44%|████▎ | 555MB / 1.27GB, 54.4MB/s  ...eckpoints/model_085000.pt: 7%|▋ | 162MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 11%|█ | 274MB / 2.49GB  ...nts/optim_085000_rank1.pt: 10%|█ | 250MB / 2.49GB  ...nts/optim_085000_rank3.pt: 10%|▉ | 246MB / 2.49GB  ...nts/optim_085000_rank7.pt: 10%|▉ | 245MB / 2.49GB  ...nts/optim_085000_rank2.pt: 10%|█ | 250MB / 2.49GB  ...nts/optim_075000_rank6.pt: 77%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_085000_rank5.pt: 10%|▉ | 241MB / 2.49GB  Processing Files (6 / 14) : 41%|████▏ | 18.3GB / 44.5GB, 1.70GB/s New Data Upload : 43%|████▎ | 575MB / 1.34GB, 56.4MB/s  ...eckpoints/model_085000.pt: 7%|▋ | 165MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 12%|█▏ | 291MB / 2.49GB  ...nts/optim_085000_rank1.pt: 11%|█ | 262MB / 2.49GB  ...nts/optim_085000_rank3.pt: 11%|█ | 262MB / 2.49GB  ...nts/optim_085000_rank7.pt: 10%|█ | 257MB / 2.49GB  ...nts/optim_085000_rank2.pt: 11%|█ | 264MB / 2.49GB  ...nts/optim_075000_rank6.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_085000_rank5.pt: 10%|█ | 256MB / 2.49GB  Processing Files (6 / 14) : 42%|████▏ | 18.6GB / 44.5GB, 1.68GB/s New Data Upload : 45%|████▍ | 598MB / 1.34GB, 58.6MB/s  ...eckpoints/model_085000.pt: 7%|▋ | 173MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 13%|█▎ | 311MB / 2.49GB  ...nts/optim_085000_rank1.pt: 11%|█▏ | 280MB / 2.49GB  ...nts/optim_085000_rank3.pt: 11%|█▏ | 282MB / 2.49GB  ...nts/optim_085000_rank7.pt: 11%|█ | 276MB / 2.49GB  ...nts/optim_085000_rank2.pt: 11%|█▏ | 283MB / 2.49GB  ...nts/optim_075000_rank6.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_085000_rank5.pt: 11%|█ | 276MB / 2.49GB  Processing Files (6 / 14) : 42%|████▏ | 18.8GB / 44.5GB, 1.65GB/s New Data Upload : 43%|████▎ | 633MB / 1.48GB, 62.1MB/s  ...eckpoints/model_085000.pt: 8%|▊ | 180MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 14%|█▎ | 336MB / 2.49GB  ...nts/optim_085000_rank1.pt: 12%|█▏ | 306MB / 2.49GB  ...nts/optim_085000_rank3.pt: 12%|█▏ | 305MB / 2.49GB  ...nts/optim_085000_rank7.pt: 12%|█▏ | 301MB / 2.49GB  ...nts/optim_085000_rank2.pt: 12%|█▏ | 305MB / 2.49GB  ...nts/optim_075000_rank6.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_085000_rank5.pt: 12%|█▏ | 299MB / 2.49GB  Processing Files (6 / 14) : 43%|████▎ | 19.1GB / 44.5GB, 1.62GB/s New Data Upload : 44%|████▎ | 672MB / 1.54GB, 65.7MB/s  ...eckpoints/model_085000.pt: 8%|▊ | 185MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 14%|█▍ | 350MB / 2.49GB  ...nts/optim_085000_rank1.pt: 13%|█▎ | 321MB / 2.49GB  ...nts/optim_085000_rank3.pt: 13%|█▎ | 317MB / 2.49GB  ...nts/optim_085000_rank7.pt: 13%|█▎ | 316MB / 2.49GB  ...nts/optim_085000_rank2.pt: 13%|█▎ | 316MB / 2.49GB  ...nts/optim_075000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 13%|█▎ | 312MB / 2.49GB  Processing Files (6 / 14) : 44%|████▎ | 19.4GB / 44.5GB, 1.59GB/s New Data Upload : 45%|████▌ | 699MB / 1.54GB, 68.3MB/s  ...eckpoints/model_085000.pt: 8%|▊ | 187MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 15%|█▍ | 371MB / 2.49GB  ...nts/optim_085000_rank1.pt: 14%|█▍ | 344MB / 2.49GB  ...nts/optim_085000_rank3.pt: 14%|█▎ | 338MB / 2.49GB  ...nts/optim_085000_rank7.pt: 14%|█▎ | 338MB / 2.49GB  ...nts/optim_085000_rank2.pt: 14%|█▎ | 338MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 13%|█▎ | 334MB / 2.49GB  Processing Files (7 / 14) : 44%|████▍ | 19.5GB / 44.5GB, 1.56GB/s New Data Upload : 47%|████▋ | 730MB / 1.54GB, 71.4MB/s  ...eckpoints/model_085000.pt: 8%|▊ | 196MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 15%|█▌ | 382MB / 2.49GB  ...nts/optim_085000_rank1.pt: 14%|█▍ | 354MB / 2.49GB  ...nts/optim_085000_rank3.pt: 14%|█▍ | 347MB / 2.49GB  ...nts/optim_085000_rank7.pt: 14%|█▍ | 348MB / 2.49GB  ...nts/optim_085000_rank2.pt: 14%|█▍ | 346MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 14%|█▍ | 343MB / 2.49GB  Processing Files (7 / 14) : 44%|████▍ | 19.6GB / 44.5GB, 1.52GB/s New Data Upload : 45%|████▍ | 753MB / 1.68GB, 73.4MB/s  ...eckpoints/model_085000.pt: 9%|▊ | 204MB / 2.35GB  ...nts/optim_075000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 16%|█▋ | 408MB / 2.49GB  ...nts/optim_085000_rank1.pt: 15%|█▌ | 382MB / 2.49GB  ...nts/optim_085000_rank3.pt: 15%|█▍ | 371MB / 2.49GB  ...nts/optim_085000_rank7.pt: 15%|█▌ | 375MB / 2.49GB  ...nts/optim_085000_rank2.pt: 15%|█▍ | 370MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  [+ 1 files] : 9%|▉ | 467MB / 4.97GB  Processing Files (7 / 15) : 45%|████▍ | 19.8GB / 44.5GB, 1.50GB/s New Data Upload : 44%|████▍ | 797MB / 1.81GB, 77.3MB/s  ...eckpoints/model_085000.pt: 9%|▉ | 219MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 17%|█▋ | 424MB / 2.49GB  ...nts/optim_085000_rank1.pt: 16%|█▌ | 396MB / 2.49GB  ...nts/optim_085000_rank3.pt: 16%|█▌ | 385MB / 2.49GB  ...nts/optim_085000_rank7.pt: 16%|█▌ | 390MB / 2.49GB  ...nts/optim_085000_rank2.pt: 15%|█▌ | 384MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 15%|█▌ | 382MB / 2.49GB  ...nts/optim_085000_rank6.pt: 8%|▊ | 196MB / 2.49GB  Processing Files (7 / 15) : 45%|████▌ | 20.0GB / 44.5GB, 1.47GB/s New Data Upload : 46%|████▌ | 831MB / 1.81GB, 79.8MB/s  ...eckpoints/model_085000.pt: 10%|▉ | 227MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 18%|█▊ | 447MB / 2.49GB  ...nts/optim_085000_rank1.pt: 17%|█▋ | 415MB / 2.49GB  ...nts/optim_085000_rank3.pt: 16%|█▋ | 408MB / 2.49GB  ...nts/optim_085000_rank7.pt: 16%|█▋ | 408MB / 2.49GB  ...nts/optim_085000_rank2.pt: 16%|█▋ | 405MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 16%|█▌ | 404MB / 2.49GB  ...nts/optim_085000_rank6.pt: 11%|█▏ | 280MB / 2.49GB  Processing Files (7 / 15) : 46%|████▌ | 20.3GB / 44.5GB, 1.44GB/s New Data Upload : 44%|████▍ | 865MB / 1.94GB, 82.8MB/s  ...eckpoints/model_085000.pt: 10%|█ | 237MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 19%|█▊ | 463MB / 2.49GB  ...nts/optim_085000_rank1.pt: 17%|█▋ | 427MB / 2.49GB  ...nts/optim_085000_rank3.pt: 17%|█▋ | 421MB / 2.49GB  ...nts/optim_085000_rank7.pt: 17%|█▋ | 421MB / 2.49GB  ...nts/optim_085000_rank2.pt: 17%|█▋ | 419MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 17%|█▋ | 417MB / 2.49GB  ...nts/optim_085000_rank6.pt: 14%|█▎ | 342MB / 2.49GB  Processing Files (7 / 15) : 46%|████▌ | 20.4GB / 44.5GB, 1.41GB/s New Data Upload : 46%|████▌ | 892MB / 1.94GB, 85.0MB/s  ...eckpoints/model_085000.pt: 11%|█ | 247MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 19%|█▉ | 483MB / 2.49GB  ...nts/optim_085000_rank1.pt: 18%|█▊ | 444MB / 2.49GB  ...nts/optim_085000_rank3.pt: 18%|█▊ | 438MB / 2.49GB  ...nts/optim_085000_rank7.pt: 18%|█▊ | 437MB / 2.49GB  ...nts/optim_085000_rank2.pt: 18%|█▊ | 436MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 17%|█▋ | 434MB / 2.49GB  ...nts/optim_085000_rank6.pt: 16%|█▌ | 387MB / 2.49GB  Processing Files (7 / 15) : 46%|████▋ | 20.6GB / 44.5GB, 1.37GB/s New Data Upload : 48%|████▊ | 925MB / 1.94GB, 87.8MB/s  ...eckpoints/model_085000.pt: 11%|█ | 255MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 20%|██ | 506MB / 2.49GB  ...nts/optim_085000_rank1.pt: 19%|█▉ | 467MB / 2.49GB  ...nts/optim_085000_rank3.pt: 18%|█▊ | 457MB / 2.49GB  ...nts/optim_085000_rank7.pt: 18%|█▊ | 459MB / 2.49GB  ...nts/optim_085000_rank2.pt: 18%|█▊ | 456MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 18%|█▊ | 455MB / 2.49GB  ...nts/optim_085000_rank6.pt: 17%|█▋ | 428MB / 2.49GB  Processing Files (7 / 15) : 47%|████▋ | 20.7GB / 44.5GB, 1.34GB/s New Data Upload : 49%|████▉ | 962MB / 1.94GB, 90.9MB/s  ...eckpoints/model_085000.pt: 11%|█ | 263MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 21%|██▏ | 530MB / 2.49GB  ...nts/optim_085000_rank1.pt: 20%|█▉ | 486MB / 2.49GB  ...nts/optim_085000_rank3.pt: 19%|█▉ | 478MB / 2.49GB  ...nts/optim_085000_rank7.pt: 19%|█▉ | 478MB / 2.49GB  ...nts/optim_085000_rank2.pt: 19%|█▉ | 476MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 19%|█▉ | 475MB / 2.49GB  ...nts/optim_085000_rank6.pt: 19%|█▉ | 467MB / 2.49GB  Processing Files (7 / 15) : 47%|████▋ | 20.9GB / 44.5GB, 1.31GB/s New Data Upload : 49%|████▉ | 995MB / 2.01GB, 93.5MB/s  ...eckpoints/model_085000.pt: 12%|█▏ | 271MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 22%|██▏ | 550MB / 2.49GB  ...nts/optim_085000_rank1.pt: 20%|██ | 505MB / 2.49GB  ...nts/optim_085000_rank3.pt: 20%|██ | 497MB / 2.49GB  ...nts/optim_085000_rank7.pt: 20%|█▉ | 497MB / 2.49GB  ...nts/optim_085000_rank2.pt: 20%|█▉ | 497MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 20%|█▉ | 494MB / 2.49GB  ...nts/optim_085000_rank6.pt: 20%|█▉ | 487MB / 2.49GB  Processing Files (7 / 15) : 47%|████▋ | 21.1GB / 44.5GB, 1.27GB/s New Data Upload : 51%|█████ | 1.03GB / 2.01GB, 96.2MB/s  ...eckpoints/model_085000.pt: 12%|█▏ | 282MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 23%|██▎ | 570MB / 2.49GB  ...nts/optim_085000_rank1.pt: 21%|██ | 520MB / 2.49GB  ...nts/optim_085000_rank3.pt: 21%|██ | 512MB / 2.49GB  ...nts/optim_085000_rank7.pt: 21%|██ | 511MB / 2.49GB  ...nts/optim_085000_rank2.pt: 21%|██ | 513MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 20%|██ | 510MB / 2.49GB  ...nts/optim_085000_rank6.pt: 20%|██ | 503MB / 2.49GB  Processing Files (7 / 15) : 48%|████▊ | 21.2GB / 44.5GB, 1.24GB/s New Data Upload : 51%|█████ | 1.06GB / 2.08GB, 98.8MB/s  ...eckpoints/model_085000.pt: 12%|█▏ | 292MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 24%|██▎ | 587MB / 2.49GB  ...nts/optim_085000_rank1.pt: 22%|██▏ | 538MB / 2.49GB  ...nts/optim_085000_rank3.pt: 21%|██▏ | 531MB / 2.49GB  ...nts/optim_085000_rank7.pt: 21%|██▏ | 530MB / 2.49GB  ...nts/optim_085000_rank2.pt: 21%|██▏ | 534MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 21%|██▏ | 529MB / 2.49GB  ...nts/optim_085000_rank6.pt: 21%|██ | 522MB / 2.49GB  Processing Files (7 / 15) : 48%|████▊ | 21.3GB / 44.5GB, 1.20GB/s New Data Upload : 51%|█████ | 1.09GB / 2.15GB, 102MB/s  ...eckpoints/model_085000.pt: 13%|█▎ | 302MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 24%|██▍ | 603MB / 2.49GB  ...nts/optim_085000_rank1.pt: 22%|██▏ | 553MB / 2.49GB  ...nts/optim_085000_rank3.pt: 22%|██▏ | 544MB / 2.49GB  ...nts/optim_085000_rank7.pt: 22%|██▏ | 543MB / 2.49GB  ...nts/optim_085000_rank2.pt: 22%|██▏ | 549MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 22%|██▏ | 543MB / 2.49GB  ...nts/optim_085000_rank6.pt: 22%|██▏ | 535MB / 2.49GB  Processing Files (7 / 15) : 48%|████▊ | 21.4GB / 44.5GB, 1.17GB/s New Data Upload : 51%|█████ | 1.13GB / 2.21GB, 105MB/s  ...eckpoints/model_085000.pt: 13%|█▎ | 304MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 25%|██▍ | 619MB / 2.49GB  ...nts/optim_085000_rank1.pt: 23%|██▎ | 574MB / 2.49GB  ...nts/optim_085000_rank3.pt: 22%|██▏ | 558MB / 2.49GB  ...nts/optim_085000_rank7.pt: 23%|██▎ | 561MB / 2.49GB  ...nts/optim_085000_rank2.pt: 23%|██▎ | 566MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 22%|██▏ | 558MB / 2.49GB  ...nts/optim_085000_rank6.pt: 22%|██▏ | 551MB / 2.49GB  Processing Files (7 / 15) : 48%|████▊ | 21.6GB / 44.5GB, 1.15GB/s New Data Upload : 51%|█████ | 1.15GB / 2.28GB, 107MB/s  ...eckpoints/model_085000.pt: 13%|█▎ | 310MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 26%|██▌ | 641MB / 2.49GB  ...nts/optim_085000_rank1.pt: 24%|██▍ | 596MB / 2.49GB  ...nts/optim_085000_rank3.pt: 23%|██▎ | 579MB / 2.49GB  ...nts/optim_085000_rank7.pt: 23%|██▎ | 584MB / 2.49GB  ...nts/optim_085000_rank2.pt: 24%|██▎ | 589MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 23%|██▎ | 578MB / 2.49GB  ...nts/optim_085000_rank6.pt: 23%|██▎ | 572MB / 2.49GB  Processing Files (7 / 15) : 49%|████▉ | 21.7GB / 44.5GB, 1.14GB/s New Data Upload : 51%|█████ | 1.20GB / 2.35GB, 111MB/s  ...eckpoints/model_085000.pt: 13%|█▎ | 314MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 26%|██▋ | 657MB / 2.49GB  ...nts/optim_085000_rank1.pt: 25%|██▍ | 619MB / 2.49GB  ...nts/optim_085000_rank3.pt: 24%|██▍ | 597MB / 2.49GB  ...nts/optim_085000_rank7.pt: 24%|██▍ | 605MB / 2.49GB  ...nts/optim_085000_rank2.pt: 25%|██▍ | 609MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 24%|██▍ | 596MB / 2.49GB  ...nts/optim_085000_rank6.pt: 24%|██▍ | 590MB / 2.49GB  Processing Files (7 / 15) : 49%|████▉ | 21.9GB / 44.5GB, 1.13GB/s New Data Upload : 53%|█████▎ | 1.24GB / 2.35GB, 115MB/s  ...eckpoints/model_085000.pt: 14%|█▎ | 319MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 27%|██▋ | 681MB / 2.49GB  ...nts/optim_085000_rank1.pt: 26%|██▌ | 639MB / 2.49GB  ...nts/optim_085000_rank3.pt: 25%|██▍ | 617MB / 2.49GB  ...nts/optim_085000_rank7.pt: 25%|██▌ | 624MB / 2.49GB  ...nts/optim_085000_rank2.pt: 26%|██▌ | 634MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 25%|██▍ | 621MB / 2.49GB  ...nts/optim_085000_rank6.pt: 25%|██▍ | 613MB / 2.49GB  Processing Files (7 / 15) : 49%|████▉ | 22.0GB / 44.5GB, 1.13GB/s New Data Upload : 53%|█████▎ | 1.28GB / 2.41GB, 119MB/s  ...eckpoints/model_085000.pt: 14%|█▍ | 328MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 28%|██▊ | 705MB / 2.49GB  ...nts/optim_085000_rank1.pt: 27%|██▋ | 663MB / 2.49GB  ...nts/optim_085000_rank3.pt: 26%|██▌ | 637MB / 2.49GB  ...nts/optim_085000_rank7.pt: 26%|██▌ | 642MB / 2.49GB  ...nts/optim_085000_rank2.pt: 26%|██▋ | 657MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 26%|██▌ | 646MB / 2.49GB  ...nts/optim_085000_rank6.pt: 26%|██▌ | 635MB / 2.49GB  Processing Files (7 / 15) : 50%|████▉ | 22.2GB / 44.5GB, 1.12GB/s New Data Upload : 55%|█████▌ | 1.33GB / 2.41GB, 124MB/s  ...eckpoints/model_085000.pt: 14%|█▍ | 333MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 29%|██▉ | 721MB / 2.49GB  ...nts/optim_085000_rank1.pt: 27%|██▋ | 681MB / 2.49GB  ...nts/optim_085000_rank3.pt: 26%|██▌ | 651MB / 2.49GB  ...nts/optim_085000_rank7.pt: 26%|██▋ | 655MB / 2.49GB  ...nts/optim_085000_rank2.pt: 27%|██▋ | 673MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 27%|██▋ | 667MB / 2.49GB  ...nts/optim_085000_rank6.pt: 26%|██▌ | 649MB / 2.49GB  Processing Files (7 / 15) : 50%|█████ | 22.3GB / 44.5GB, 1.11GB/s New Data Upload : 57%|█████▋ | 1.37GB / 2.41GB, 127MB/s  ...eckpoints/model_085000.pt: 14%|█▍ | 338MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 30%|██▉ | 745MB / 2.49GB  ...nts/optim_085000_rank1.pt: 29%|██▊ | 708MB / 2.49GB  ...nts/optim_085000_rank3.pt: 27%|██▋ | 676MB / 2.49GB  ...nts/optim_085000_rank7.pt: 27%|██▋ | 680MB / 2.49GB  ...nts/optim_085000_rank2.pt: 28%|██▊ | 698MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 28%|██▊ | 692MB / 2.49GB  ...nts/optim_085000_rank6.pt: 27%|██▋ | 675MB / 2.49GB  Processing Files (7 / 15) : 51%|█████ | 22.5GB / 44.5GB, 1.10GB/s New Data Upload : 58%|█████▊ | 1.41GB / 2.41GB, 132MB/s  ...eckpoints/model_085000.pt: 15%|█▍ | 341MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 31%|███ | 770MB / 2.49GB  ...nts/optim_085000_rank1.pt: 29%|██▉ | 729MB / 2.49GB  ...nts/optim_085000_rank3.pt: 28%|██▊ | 698MB / 2.49GB  ...nts/optim_085000_rank7.pt: 28%|██▊ | 701MB / 2.49GB  ...nts/optim_085000_rank2.pt: 29%|██▉ | 724MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 29%|██▉ | 715MB / 2.49GB  ...nts/optim_085000_rank6.pt: 28%|██▊ | 699MB / 2.49GB  Processing Files (7 / 15) : 51%|█████ | 22.6GB / 44.5GB, 1.09GB/s New Data Upload : 57%|█████▋ | 1.45GB / 2.55GB, 135MB/s  ...eckpoints/model_085000.pt: 15%|█▍ | 349MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 32%|███▏ | 797MB / 2.49GB  ...nts/optim_085000_rank1.pt: 31%|███ | 760MB / 2.49GB  ...nts/optim_085000_rank3.pt: 29%|██▉ | 724MB / 2.49GB  ...nts/optim_085000_rank7.pt: 29%|██▉ | 726MB / 2.49GB  ...nts/optim_085000_rank2.pt: 30%|███ | 756MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 30%|██▉ | 746MB / 2.49GB  ...nts/optim_085000_rank6.pt: 29%|██▉ | 726MB / 2.49GB  Processing Files (7 / 15) : 51%|█████▏ | 22.8GB / 44.5GB, 1.09GB/s New Data Upload : 56%|█████▌ | 1.50GB / 2.68GB, 140MB/s  ...eckpoints/model_085000.pt: 15%|█▌ | 353MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 33%|███▎ | 827MB / 2.49GB  ...nts/optim_085000_rank1.pt: 32%|███▏ | 791MB / 2.49GB  ...nts/optim_085000_rank3.pt: 30%|███ | 752MB / 2.49GB  ...nts/optim_085000_rank7.pt: 30%|███ | 755MB / 2.49GB  ...nts/optim_085000_rank2.pt: 32%|███▏ | 790MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 31%|███ | 777MB / 2.49GB  ...nts/optim_085000_rank6.pt: 30%|███ | 757MB / 2.49GB  Processing Files (7 / 15) : 52%|█████▏ | 23.1GB / 44.5GB, 1.08GB/s New Data Upload : 56%|█████▋ | 1.55GB / 2.75GB, 144MB/s  ...eckpoints/model_085000.pt: 15%|█▌ | 356MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 34%|███▍ | 856MB / 2.49GB  ...nts/optim_085000_rank1.pt: 33%|███▎ | 821MB / 2.49GB  ...nts/optim_085000_rank3.pt: 31%|███▏ | 781MB / 2.49GB  ...nts/optim_085000_rank7.pt: 32%|███▏ | 783MB / 2.49GB  ...nts/optim_085000_rank2.pt: 33%|███▎ | 820MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 32%|███▏ | 803MB / 2.49GB  ...nts/optim_085000_rank6.pt: 32%|███▏ | 787MB / 2.49GB  Processing Files (7 / 15) : 52%|█████▏ | 23.3GB / 44.5GB, 1.08GB/s New Data Upload : 57%|█████▋ | 1.60GB / 2.82GB, 148MB/s  ...eckpoints/model_085000.pt: 15%|█▌ | 362MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 36%|███▌ | 885MB / 2.49GB  ...nts/optim_085000_rank1.pt: 34%|███▍ | 852MB / 2.49GB  ...nts/optim_085000_rank3.pt: 33%|███▎ | 811MB / 2.49GB  ...nts/optim_085000_rank7.pt: 33%|███▎ | 814MB / 2.49GB  ...nts/optim_085000_rank2.pt: 34%|███▍ | 849MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 34%|███▎ | 835MB / 2.49GB  ...nts/optim_085000_rank6.pt: 33%|███▎ | 818MB / 2.49GB  Processing Files (7 / 15) : 53%|█████▎ | 23.5GB / 44.5GB, 1.07GB/s New Data Upload : 57%|█████▋ | 1.65GB / 2.88GB, 152MB/s  ...eckpoints/model_085000.pt: 16%|█▌ | 364MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 37%|███▋ | 913MB / 2.49GB  ...nts/optim_085000_rank1.pt: 35%|███▌ | 880MB / 2.49GB  ...nts/optim_085000_rank3.pt: 34%|███▍ | 839MB / 2.49GB  ...nts/optim_085000_rank7.pt: 34%|███▍ | 841MB / 2.49GB  ...nts/optim_085000_rank2.pt: 35%|███▌ | 879MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 35%|███▍ | 863MB / 2.49GB  ...nts/optim_085000_rank6.pt: 34%|███▍ | 847MB / 2.49GB  Processing Files (7 / 15) : 53%|█████▎ | 23.7GB / 44.5GB, 1.06GB/s New Data Upload : 58%|█████▊ | 1.69GB / 2.88GB, 154MB/s  ...eckpoints/model_085000.pt: 16%|█▌ | 370MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 38%|███▊ | 946MB / 2.49GB  ...nts/optim_085000_rank1.pt: 37%|███▋ | 913MB / 2.49GB  ...nts/optim_085000_rank3.pt: 35%|███▌ | 872MB / 2.49GB  ...nts/optim_085000_rank7.pt: 35%|███▌ | 874MB / 2.49GB  ...nts/optim_085000_rank2.pt: 37%|███▋ | 909MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 36%|███▌ | 898MB / 2.49GB  ...nts/optim_085000_rank6.pt: 35%|███▌ | 880MB / 2.49GB  Processing Files (7 / 15) : 54%|█████▍ | 23.9GB / 44.5GB, 1.05GB/s New Data Upload : 60%|██████ | 1.74GB / 2.88GB, 158MB/s  ...eckpoints/model_085000.pt: 16%|█▌ | 372MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 39%|███▉ | 971MB / 2.49GB  ...nts/optim_085000_rank1.pt: 38%|███▊ | 938MB / 2.49GB  ...nts/optim_085000_rank3.pt: 36%|███▌ | 896MB / 2.49GB  ...nts/optim_085000_rank7.pt: 36%|███▌ | 899MB / 2.49GB  ...nts/optim_085000_rank2.pt: 37%|███▋ | 932MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 37%|███▋ | 924MB / 2.49GB  ...nts/optim_085000_rank6.pt: 36%|███▋ | 905MB / 2.49GB  Processing Files (7 / 15) : 54%|█████▍ | 24.1GB / 44.5GB, 1.04GB/s New Data Upload : 60%|██████ | 1.77GB / 2.95GB, 160MB/s  ...eckpoints/model_085000.pt: 16%|█▌ | 376MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 40%|████ | 999MB / 2.49GB  ...nts/optim_085000_rank1.pt: 39%|███▊ | 962MB / 2.49GB  ...nts/optim_085000_rank3.pt: 37%|███▋ | 925MB / 2.49GB  ...nts/optim_085000_rank7.pt: 37%|███▋ | 927MB / 2.49GB  ...nts/optim_085000_rank2.pt: 39%|███▊ | 958MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 38%|███▊ | 951MB / 2.49GB  ...nts/optim_085000_rank6.pt: 38%|███▊ | 934MB / 2.49GB  Processing Files (7 / 15) : 55%|█████▍ | 24.3GB / 44.5GB, 1.03GB/s New Data Upload : 61%|██████▏ | 1.81GB / 2.95GB, 162MB/s  ...eckpoints/model_085000.pt: 16%|█▌ | 378MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_085000_rank1.pt: 40%|███▉ | 985MB / 2.49GB  ...nts/optim_085000_rank3.pt: 38%|███▊ | 944MB / 2.49GB  ...nts/optim_085000_rank7.pt: 38%|███▊ | 947MB / 2.49GB  ...nts/optim_085000_rank2.pt: 39%|███▉ | 979MB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 39%|███▉ | 972MB / 2.49GB  ...nts/optim_085000_rank6.pt: 38%|███▊ | 955MB / 2.49GB  Processing Files (7 / 15) : 55%|█████▍ | 24.4GB / 44.5GB, 1.02GB/s New Data Upload : 61%|██████ | 1.84GB / 3.02GB, 164MB/s  ...eckpoints/model_085000.pt: 16%|█▋ | 382MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_085000_rank1.pt: 40%|████ | 1.01GB / 2.49GB  ...nts/optim_085000_rank3.pt: 39%|███▉ | 970MB / 2.49GB  ...nts/optim_085000_rank7.pt: 39%|███▉ | 969MB / 2.49GB  ...nts/optim_085000_rank2.pt: 40%|████ | 1.00GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 40%|████ | 995MB / 2.49GB  ...nts/optim_085000_rank6.pt: 39%|███▉ | 980MB / 2.49GB  Processing Files (7 / 15) : 55%|█████▌ | 24.6GB / 44.5GB, 1.00GB/s New Data Upload : 63%|██████▎ | 1.89GB / 3.02GB, 166MB/s  ...eckpoints/model_085000.pt: 16%|█▋ | 385MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_085000_rank1.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_085000_rank3.pt: 40%|███▉ | 990MB / 2.49GB  ...nts/optim_085000_rank7.pt: 40%|███▉ | 990MB / 2.49GB  ...nts/optim_085000_rank2.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_085000_rank6.pt: 40%|████ | 1.00GB / 2.49GB  Processing Files (7 / 15) : 56%|█████▌ | 24.7GB / 44.5GB, 984MB/s New Data Upload : 64%|██████▎ | 1.92GB / 3.02GB, 167MB/s  ...eckpoints/model_085000.pt: 17%|█▋ | 389MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 44%|████▍ | 1.09GB / 2.49GB  ...nts/optim_085000_rank1.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_085000_rank3.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_085000_rank7.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_085000_rank2.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_085000_rank6.pt: 41%|████ | 1.02GB / 2.49GB  Processing Files (7 / 15) : 56%|█████▌ | 24.9GB / 44.5GB, 969MB/s New Data Upload : 63%|██████▎ | 1.95GB / 3.08GB, 169MB/s  ...eckpoints/model_085000.pt: 17%|█▋ | 390MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 45%|████▍ | 1.12GB / 2.49GB  ...nts/optim_085000_rank1.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_085000_rank3.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_085000_rank7.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_085000_rank2.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_085000_rank6.pt: 42%|████▏ | 1.05GB / 2.49GB  Processing Files (7 / 15) : 56%|█████▋ | 25.1GB / 44.5GB, 961MB/s New Data Upload : 62%|██████▏ | 1.99GB / 3.22GB, 170MB/s  ...eckpoints/model_085000.pt: 17%|█▋ | 392MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 46%|████▌ | 1.13GB / 2.49GB  ...nts/optim_085000_rank1.pt: 44%|████▎ | 1.08GB / 2.49GB  ...nts/optim_085000_rank3.pt: 42%|████▏ | 1.06GB / 2.49GB  ...nts/optim_085000_rank7.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_085000_rank2.pt: 44%|████▎ | 1.08GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 43%|████▎ | 1.08GB / 2.49GB  ...nts/optim_085000_rank6.pt: 43%|████▎ | 1.07GB / 2.49GB  Processing Files (7 / 15) : 57%|█████▋ | 25.2GB / 44.5GB, 954MB/s New Data Upload : 61%|██████▏ | 2.02GB / 3.29GB, 171MB/s  ...eckpoints/model_085000.pt: 17%|█▋ | 395MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_085000_rank1.pt: 44%|████▍ | 1.11GB / 2.49GB  ...nts/optim_085000_rank3.pt: 44%|████▎ | 1.08GB / 2.49GB  ...nts/optim_085000_rank7.pt: 44%|████▎ | 1.09GB / 2.49GB  ...nts/optim_085000_rank2.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_085000_rank6.pt: 44%|████▍ | 1.09GB / 2.49GB  Processing Files (7 / 15) : 57%|█████▋ | 25.4GB / 44.5GB, 942MB/s New Data Upload : 63%|██████▎ | 2.06GB / 3.29GB, 173MB/s  ...eckpoints/model_085000.pt: 17%|█▋ | 401MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 48%|████▊ | 1.18GB / 2.49GB  ...nts/optim_085000_rank1.pt: 45%|████▌ | 1.13GB / 2.49GB  ...nts/optim_085000_rank3.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_085000_rank7.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_085000_rank2.pt: 45%|████▌ | 1.12GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 45%|████▌ | 1.12GB / 2.49GB  ...nts/optim_085000_rank6.pt: 45%|████▍ | 1.11GB / 2.49GB  Processing Files (7 / 15) : 57%|█████▋ | 25.5GB / 44.5GB, 934MB/s New Data Upload : 63%|██████▎ | 2.10GB / 3.35GB, 174MB/s  ...eckpoints/model_085000.pt: 17%|█▋ | 403MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 49%|████▊ | 1.21GB / 2.49GB  ...nts/optim_085000_rank1.pt: 46%|████▋ | 1.15GB / 2.49GB  ...nts/optim_085000_rank3.pt: 45%|████▌ | 1.13GB / 2.49GB  ...nts/optim_085000_rank7.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_085000_rank2.pt: 46%|████▌ | 1.15GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 46%|████▌ | 1.15GB / 2.49GB  ...nts/optim_085000_rank6.pt: 46%|████▌ | 1.14GB / 2.49GB  Processing Files (7 / 15) : 58%|█████▊ | 25.7GB / 44.5GB, 930MB/s New Data Upload : 64%|██████▍ | 2.14GB / 3.35GB, 175MB/s  ...eckpoints/model_085000.pt: 17%|█▋ | 407MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_085000_rank1.pt: 47%|████▋ | 1.18GB / 2.49GB  ...nts/optim_085000_rank3.pt: 46%|████▋ | 1.15GB / 2.49GB  ...nts/optim_085000_rank7.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_085000_rank2.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_085000_rank6.pt: 47%|████▋ | 1.16GB / 2.49GB  Processing Files (7 / 15) : 58%|█████▊ | 25.9GB / 44.5GB, 919MB/s New Data Upload : 65%|██████▌ | 2.18GB / 3.35GB, 175MB/s  ...eckpoints/model_085000.pt: 18%|█▊ | 412MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_085000_rank1.pt: 49%|████▊ | 1.21GB / 2.49GB  ...nts/optim_085000_rank3.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_085000_rank7.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_085000_rank2.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_085000_rank6.pt: 48%|████▊ | 1.20GB / 2.49GB  Processing Files (7 / 15) : 59%|█████▉ | 26.1GB / 44.5GB, 919MB/s New Data Upload : 65%|██████▌ | 2.23GB / 3.42GB, 178MB/s  ...eckpoints/model_085000.pt: 18%|█▊ | 416MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_085000_rank1.pt: 49%|████▉ | 1.23GB / 2.49GB  ...nts/optim_085000_rank3.pt: 49%|████▊ | 1.21GB / 2.49GB  ...nts/optim_085000_rank7.pt: 49%|████▉ | 1.23GB / 2.49GB  ...nts/optim_085000_rank2.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 49%|████▉ | 1.23GB / 2.49GB  ...nts/optim_085000_rank6.pt: 49%|████▉ | 1.22GB / 2.49GB  Processing Files (7 / 15) : 59%|█████▉ | 26.3GB / 44.5GB, 907MB/s New Data Upload : 65%|██████▍ | 2.26GB / 3.49GB, 179MB/s  ...eckpoints/model_085000.pt: 18%|█▊ | 420MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 53%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_085000_rank1.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_085000_rank3.pt: 50%|█████ | 1.24GB / 2.49GB  ...nts/optim_085000_rank7.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_085000_rank2.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_085000_rank6.pt: 50%|█████ | 1.25GB / 2.49GB  Processing Files (7 / 15) : 60%|█████▉ | 26.5GB / 44.5GB, 905MB/s New Data Upload : 65%|██████▌ | 2.31GB / 3.55GB, 181MB/s  ...eckpoints/model_085000.pt: 18%|█▊ | 425MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_085000_rank1.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_085000_rank3.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_085000_rank7.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_085000_rank2.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_085000_rank6.pt: 51%|█████ | 1.27GB / 2.49GB  Processing Files (7 / 15) : 60%|██████ | 26.7GB / 44.5GB, 898MB/s New Data Upload : 61%|██████▏ | 2.35GB / 3.82GB, 182MB/s  ...eckpoints/model_085000.pt: 18%|█▊ | 426MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_085000_rank1.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_085000_rank3.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_085000_rank7.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_085000_rank2.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_085000_rank6.pt: 52%|█████▏ | 1.30GB / 2.49GB  Processing Files (7 / 15) : 60%|██████ | 26.9GB / 44.5GB, 896MB/s New Data Upload : 61%|██████▏ | 2.39GB / 3.89GB, 184MB/s  ...eckpoints/model_085000.pt: 18%|█▊ | 431MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...nts/optim_085000_rank1.pt: 53%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_085000_rank3.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_085000_rank7.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_085000_rank2.pt: 53%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 53%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_085000_rank6.pt: 53%|█████▎ | 1.32GB / 2.49GB  Processing Files (7 / 15) : 61%|██████ | 27.1GB / 44.5GB, 895MB/s New Data Upload : 63%|██████▎ | 2.43GB / 3.89GB, 186MB/s  ...eckpoints/model_085000.pt: 19%|█▊ | 437MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_085000_rank1.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_085000_rank3.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_085000_rank7.pt: 55%|█████▍ | 1.37GB / 2.49GB  ...nts/optim_085000_rank2.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_085000_rank6.pt: 54%|█████▍ | 1.34GB / 2.49GB  Processing Files (7 / 15) : 61%|██████ | 27.2GB / 44.5GB, 890MB/s New Data Upload : 63%|██████▎ | 2.47GB / 3.96GB, 188MB/s  ...eckpoints/model_085000.pt: 19%|█▉ | 441MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_085000_rank1.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_085000_rank3.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_085000_rank7.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_085000_rank2.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_085000_rank6.pt: 55%|█████▌ | 1.37GB / 2.49GB  Processing Files (7 / 15) : 62%|██████▏ | 27.4GB / 44.5GB, 891MB/s New Data Upload : 63%|██████▎ | 2.52GB / 4.02GB, 191MB/s  ...eckpoints/model_085000.pt: 19%|█▉ | 454MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 59%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_085000_rank1.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_085000_rank3.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_085000_rank7.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_085000_rank2.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_085000_rank6.pt: 56%|█████▌ | 1.40GB / 2.49GB  Processing Files (7 / 15) : 62%|██████▏ | 27.6GB / 44.5GB, 887MB/s New Data Upload : 64%|██████▍ | 2.57GB / 4.02GB, 193MB/s  ...eckpoints/model_085000.pt: 20%|█▉ | 463MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_085000_rank1.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_085000_rank3.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_085000_rank7.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_085000_rank2.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_085000_rank6.pt: 57%|█████▋ | 1.42GB / 2.49GB  Processing Files (7 / 15) : 63%|██████▎ | 27.8GB / 44.5GB, 879MB/s New Data Upload : 64%|██████▍ | 2.61GB / 4.09GB, 194MB/s  ...eckpoints/model_085000.pt: 20%|██ | 474MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 61%|██████ | 1.52GB / 2.49GB  ...nts/optim_085000_rank1.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_085000_rank3.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_085000_rank7.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_085000_rank2.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_085000_rank6.pt: 58%|█████▊ | 1.45GB / 2.49GB  Processing Files (7 / 15) : 63%|██████▎ | 28.0GB / 44.5GB, 870MB/s New Data Upload : 64%|██████▍ | 2.66GB / 4.16GB, 195MB/s  ...eckpoints/model_085000.pt: 20%|██ | 478MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_085000_rank1.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 59%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_085000_rank7.pt: 61%|██████ | 1.52GB / 2.49GB  ...nts/optim_085000_rank2.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 60%|██████ | 1.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 60%|█████▉ | 1.48GB / 2.49GB  Processing Files (7 / 15) : 64%|██████▎ | 28.3GB / 44.5GB, 871MB/s New Data Upload : 64%|██████▍ | 2.71GB / 4.22GB, 197MB/s  ...eckpoints/model_085000.pt: 21%|██ | 482MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 64%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_085000_rank1.pt: 61%|██████ | 1.52GB / 2.49GB  ...nts/optim_085000_rank3.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_085000_rank7.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_085000_rank2.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 61%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_085000_rank6.pt: 61%|██████ | 1.52GB / 2.49GB  Processing Files (7 / 15) : 64%|██████▍ | 28.5GB / 44.5GB, 879MB/s New Data Upload : 63%|██████▎ | 2.76GB / 4.36GB, 199MB/s  ...eckpoints/model_085000.pt: 21%|██ | 485MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_085000_rank1.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_085000_rank3.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_085000_rank7.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_085000_rank2.pt: 63%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_085000_rank6.pt: 62%|██████▏ | 1.55GB / 2.49GB  Processing Files (7 / 15) : 65%|██████▍ | 28.7GB / 44.5GB, 896MB/s New Data Upload : 63%|██████▎ | 2.81GB / 4.49GB, 202MB/s  ...eckpoints/model_085000.pt: 21%|██ | 491MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_085000_rank1.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_085000_rank3.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_085000_rank7.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_085000_rank2.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_085000_rank6.pt: 63%|██████▎ | 1.57GB / 2.49GB  Processing Files (7 / 15) : 65%|██████▌ | 28.9GB / 44.5GB, 889MB/s New Data Upload : 63%|██████▎ | 2.86GB / 4.56GB, 202MB/s  ...eckpoints/model_085000.pt: 21%|██ | 498MB / 2.35GB  ...nts/optim_075000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank4.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_085000_rank1.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_085000_rank3.pt: 64%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_085000_rank7.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_085000_rank2.pt: 65%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 65%|██████▍ | 1.61GB / 2.49GB  [+ 1 files] : 32%|███▏ | 1.61GB / 4.97GB  Processing Files (7 / 16) : 65%|██████▌ | 29.1GB / 44.5GB, 886MB/s New Data Upload : 63%|██████▎ | 2.90GB / 4.63GB, 202MB/s  ...eckpoints/model_085000.pt: 22%|██▏ | 514MB / 2.35GB  ...nts/optim_085000_rank4.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_085000_rank1.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_085000_rank3.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_085000_rank7.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_085000_rank2.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_085000_rank6.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_075000_rank4.pt: 6%|▋ | 160MB / 2.49GB  Processing Files (7 / 16) : 66%|██████▌ | 29.4GB / 44.5GB, 898MB/s New Data Upload : 64%|██████▎ | 2.95GB / 4.63GB, 204MB/s  ...eckpoints/model_085000.pt: 22%|██▏ | 522MB / 2.35GB  ...nts/optim_085000_rank4.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_085000_rank1.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_085000_rank3.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_085000_rank7.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_085000_rank2.pt: 67%|██████▋ | 1.68GB / 2.49GB  ...nts/optim_075000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 67%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_085000_rank6.pt: 66%|██████▌ | 1.64GB / 2.49GB  [+ 1 files] : 8%|▊ | 400MB / 4.97GB  Processing Files (7 / 17) : 67%|██████▋ | 29.8GB / 44.5GB, 922MB/s New Data Upload : 65%|██████▍ | 2.99GB / 4.63GB, 206MB/s  ...eckpoints/model_085000.pt: 23%|██▎ | 531MB / 2.35GB  ...nts/optim_085000_rank4.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_085000_rank1.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_085000_rank3.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_085000_rank7.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_085000_rank2.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_085000_rank5.pt: 67%|██████▋ | 1.68GB / 2.49GB  ...nts/optim_085000_rank6.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_075000_rank4.pt: 17%|█▋ | 416MB / 2.49GB  ...nts/optim_075000_rank3.pt: 10%|▉ | 240MB / 2.49GB  Processing Files (7 / 17) : 68%|██████▊ | 30.2GB / 44.5GB, 946MB/s New Data Upload : 65%|██████▌ | 3.03GB / 4.63GB, 206MB/s  ...eckpoints/model_085000.pt: 23%|██▎ | 544MB / 2.35GB  ...nts/optim_085000_rank4.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_085000_rank1.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_085000_rank3.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_085000_rank7.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_085000_rank2.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_085000_rank5.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_085000_rank6.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_075000_rank4.pt: 22%|██▏ | 536MB / 2.49GB  ...nts/optim_075000_rank3.pt: 15%|█▍ | 368MB / 2.49GB  Processing Files (7 / 17) : 69%|██████▉ | 30.6GB / 44.5GB, 970MB/s New Data Upload : 67%|██████▋ | 3.08GB / 4.63GB, 208MB/s  ...eckpoints/model_085000.pt: 24%|██▍ | 558MB / 2.35GB  ...nts/optim_085000_rank4.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_085000_rank1.pt: 69%|██████▊ | 1.71GB / 2.49GB  ...nts/optim_085000_rank3.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_085000_rank7.pt: 71%|███████▏ | 1.77GB / 2.49GB  ...nts/optim_085000_rank2.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_085000_rank5.pt: 69%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_085000_rank6.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_075000_rank4.pt: 27%|██▋ | 664MB / 2.49GB  [+ 1 files] : 12%|█▏ | 603MB / 4.97GB  Processing Files (7 / 18) : 70%|███████ | 31.2GB / 44.5GB, 1.01GB/s New Data Upload : 67%|██████▋ | 3.13GB / 4.69GB, 209MB/s  ...eckpoints/model_085000.pt: 24%|██▍ | 566MB / 2.35GB  ...nts/optim_085000_rank4.pt: 71%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_085000_rank1.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_085000_rank3.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_085000_rank7.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_085000_rank2.pt: 71%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_085000_rank5.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_085000_rank6.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_075000_rank4.pt: 32%|███▏ | 792MB / 2.49GB  [+ 1 files] : 17%|█▋ | 835MB / 4.97GB  Processing Files (7 / 18) : 71%|███████▏ | 31.8GB / 44.5GB, 1.05GB/s New Data Upload : 67%|██████▋ | 3.18GB / 4.76GB, 211MB/s  ...eckpoints/model_085000.pt: 24%|██▍ | 570MB / 2.35GB  ...nts/optim_085000_rank4.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_085000_rank1.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_085000_rank3.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_085000_rank7.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_085000_rank2.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_085000_rank5.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_085000_rank6.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_075000_rank4.pt: 36%|███▋ | 904MB / 2.49GB  [+ 1 files] : 22%|██▏ | 1.09GB / 4.97GB  Processing Files (7 / 18) : 73%|███████▎ | 32.2GB / 44.5GB, 1.08GB/s New Data Upload : 67%|██████▋ | 3.21GB / 4.76GB, 211MB/s  ...eckpoints/model_085000.pt: 25%|██▍ | 578MB / 2.35GB  ...nts/optim_085000_rank4.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_085000_rank1.pt: 71%|███████▏ | 1.77GB / 2.49GB  ...nts/optim_085000_rank3.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_085000_rank7.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_085000_rank2.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_085000_rank5.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_085000_rank6.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_075000_rank4.pt: 41%|████ | 1.02GB / 2.49GB  [+ 1 files] : 27%|██▋ | 1.33GB / 4.97GB  Processing Files (7 / 18) : 74%|███████▎ | 32.8GB / 44.5GB, 1.12GB/s New Data Upload : 68%|██████▊ | 3.26GB / 4.76GB, 212MB/s  ...eckpoints/model_085000.pt: 25%|██▍ | 584MB / 2.35GB  ...nts/optim_085000_rank4.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_085000_rank1.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_085000_rank3.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_085000_rank7.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_085000_rank2.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_085000_rank5.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_085000_rank6.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_075000_rank4.pt: 46%|████▌ | 1.14GB / 2.49GB  [+ 1 files] : 31%|███ | 1.53GB / 4.97GB  Processing Files (7 / 18) : 75%|███████▍ | 33.2GB / 44.5GB, 1.16GB/s New Data Upload : 69%|██████▉ | 3.30GB / 4.76GB, 213MB/s  ...eckpoints/model_085000.pt: 25%|██▌ | 590MB / 2.35GB  ...nts/optim_085000_rank4.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_085000_rank1.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_085000_rank3.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_085000_rank7.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_085000_rank2.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_085000_rank5.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_085000_rank6.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_075000_rank4.pt: 51%|█████ | 1.26GB / 2.49GB  [+ 1 files] : 35%|███▍ | 1.74GB / 4.97GB  Processing Files (7 / 18) : 76%|███████▌ | 33.7GB / 44.5GB, 1.20GB/s New Data Upload : 70%|███████ | 3.35GB / 4.76GB, 215MB/s  ...eckpoints/model_085000.pt: 25%|██▌ | 594MB / 2.35GB  ...nts/optim_085000_rank4.pt: 75%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_085000_rank1.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_085000_rank3.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_085000_rank7.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_085000_rank2.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_085000_rank5.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_085000_rank6.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_075000_rank4.pt: 55%|█████▌ | 1.38GB / 2.49GB  [+ 1 files] : 39%|███▉ | 1.95GB / 4.97GB  Processing Files (7 / 18) : 77%|███████▋ | 34.2GB / 44.5GB, 1.22GB/s New Data Upload : 70%|██████▉ | 3.37GB / 4.83GB, 213MB/s  ...eckpoints/model_085000.pt: 26%|██▌ | 601MB / 2.35GB  ...nts/optim_085000_rank4.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_085000_rank1.pt: 75%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_085000_rank3.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_085000_rank7.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_085000_rank2.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_085000_rank5.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_085000_rank6.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_075000_rank4.pt: 60%|██████ | 1.50GB / 2.49GB  [+ 1 files] : 45%|████▍ | 2.22GB / 4.97GB  Processing Files (7 / 18) : 78%|███████▊ | 34.7GB / 44.5GB, 1.26GB/s New Data Upload : 71%|███████ | 3.41GB / 4.83GB, 214MB/s  ...eckpoints/model_085000.pt: 26%|██▌ | 612MB / 2.35GB  ...nts/optim_085000_rank4.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_085000_rank1.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_085000_rank3.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_085000_rank7.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_085000_rank2.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_085000_rank5.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_085000_rank6.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_075000_rank4.pt: 65%|██████▌ | 1.62GB / 2.49GB  [+ 1 files] : 49%|████▉ | 2.44GB / 4.97GB  Processing Files (7 / 18) : 79%|███████▉ | 35.2GB / 44.5GB, 1.29GB/s New Data Upload : 71%|███████ | 3.46GB / 4.89GB, 213MB/s  ...eckpoints/model_085000.pt: 27%|██▋ | 623MB / 2.35GB  ...nts/optim_085000_rank4.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_085000_rank1.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_085000_rank3.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_085000_rank7.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_085000_rank2.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_085000_rank5.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_085000_rank6.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_075000_rank4.pt: 70%|██████▉ | 1.74GB / 2.49GB  [+ 1 files] : 54%|█████▍ | 2.68GB / 4.97GB  Processing Files (7 / 18) : 80%|████████ | 35.7GB / 44.5GB, 1.32GB/s New Data Upload : 71%|███████ | 3.50GB / 4.96GB, 213MB/s  ...eckpoints/model_085000.pt: 27%|██▋ | 632MB / 2.35GB  ...nts/optim_085000_rank4.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_085000_rank1.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_085000_rank3.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_085000_rank7.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_085000_rank2.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_085000_rank5.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_085000_rank6.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_075000_rank4.pt: 75%|███████▍ | 1.86GB / 2.49GB  [+ 1 files] : 59%|█████▊ | 2.91GB / 4.97GB  Processing Files (7 / 18) : 81%|████████▏ | 36.2GB / 44.5GB, 1.36GB/s New Data Upload : 71%|███████▏ | 3.54GB / 4.96GB, 213MB/s  ...eckpoints/model_085000.pt: 28%|██▊ | 647MB / 2.35GB  ...nts/optim_085000_rank4.pt: 77%|███████▋ | 1.93GB / 2.49GB  ...nts/optim_085000_rank1.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_085000_rank3.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_085000_rank7.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_085000_rank2.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_085000_rank5.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_085000_rank6.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_075000_rank4.pt: 80%|███████▉ | 1.98GB / 2.49GB  [+ 1 files] : 63%|██████▎ | 3.13GB / 4.97GB  Processing Files (7 / 18) : 82%|████████▏ | 36.6GB / 44.5GB, 1.39GB/s New Data Upload : 71%|███████▏ | 3.59GB / 5.03GB, 213MB/s  ...eckpoints/model_085000.pt: 28%|██▊ | 666MB / 2.35GB  ...nts/optim_085000_rank4.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_085000_rank1.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_085000_rank3.pt: 77%|███████▋ | 1.93GB / 2.49GB  ...nts/optim_085000_rank7.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_085000_rank2.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_085000_rank5.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_085000_rank6.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_075000_rank4.pt: 86%|████████▋ | 2.14GB / 2.49GB  [+ 1 files] : 68%|██████▊ | 3.39GB / 4.97GB  Processing Files (7 / 18) : 84%|████████▎ | 37.2GB / 44.5GB, 1.42GB/s New Data Upload : 72%|███████▏ | 3.63GB / 5.03GB, 214MB/s  ...eckpoints/model_085000.pt: 29%|██▉ | 683MB / 2.35GB  ...nts/optim_085000_rank4.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_085000_rank1.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_085000_rank3.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_085000_rank7.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_085000_rank2.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_085000_rank5.pt: 80%|███████▉ | 1.99GB / 2.49GB  ...nts/optim_085000_rank6.pt: 79%|███████▊ | 1.96GB / 2.49GB  ...nts/optim_075000_rank4.pt: 94%|█████████▎| 2.33GB / 2.49GB  [+ 1 files] : 76%|███████▌ | 3.77GB / 4.97GB  Processing Files (7 / 18) : 85%|████████▌ | 37.9GB / 44.5GB, 1.47GB/s New Data Upload : 72%|███████▏ | 3.67GB / 5.09GB, 213MB/s  ...eckpoints/model_085000.pt: 30%|███ | 711MB / 2.35GB  ...nts/optim_085000_rank4.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_085000_rank1.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_085000_rank3.pt: 79%|███████▊ | 1.96GB / 2.49GB  ...nts/optim_085000_rank7.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_085000_rank2.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_085000_rank5.pt: 81%|████████ | 2.00GB / 2.49GB  ...nts/optim_085000_rank6.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_075000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  [+ 1 files] : 83%|████████▎ | 4.12GB / 4.97GB  Processing Files (8 / 18) : 87%|████████▋ | 38.5GB / 44.5GB, 1.51GB/s New Data Upload : 72%|███████▏ | 3.73GB / 5.16GB, 213MB/s  ...eckpoints/model_085000.pt: 31%|███ | 734MB / 2.35GB  ...nts/optim_085000_rank4.pt: 79%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_085000_rank1.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_085000_rank3.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_085000_rank7.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_085000_rank2.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_085000_rank5.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_085000_rank6.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 71%|███████ | 1.76GB / 2.49GB  Processing Files (9 / 18) : 87%|████████▋ | 38.7GB / 44.5GB, 1.52GB/s New Data Upload : 73%|███████▎ | 3.77GB / 5.16GB, 213MB/s  ...eckpoints/model_085000.pt: 32%|███▏ | 750MB / 2.35GB  ...nts/optim_085000_rank4.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_085000_rank1.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_085000_rank3.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_085000_rank7.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_085000_rank2.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_085000_rank5.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_085000_rank6.pt: 81%|████████ | 2.00GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 76%|███████▌ | 1.89GB / 2.49GB  Processing Files (9 / 18) : 88%|████████▊ | 39.0GB / 44.5GB, 1.52GB/s New Data Upload : 73%|███████▎ | 3.82GB / 5.23GB, 213MB/s  ...eckpoints/model_085000.pt: 33%|███▎ | 786MB / 2.35GB  ...nts/optim_085000_rank4.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_085000_rank1.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_085000_rank3.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_085000_rank7.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_085000_rank2.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_085000_rank5.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_085000_rank6.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 80%|███████▉ | 1.98GB / 2.49GB  Processing Files (9 / 18) : 88%|████████▊ | 39.3GB / 44.5GB, 1.53GB/s New Data Upload : 73%|███████▎ | 3.88GB / 5.30GB, 215MB/s  ...eckpoints/model_085000.pt: 35%|███▍ | 814MB / 2.35GB  ...nts/optim_085000_rank4.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_085000_rank1.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_085000_rank3.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_085000_rank7.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_085000_rank2.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_085000_rank5.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_085000_rank6.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 81%|████████▏ | 2.02GB / 2.49GB  Processing Files (9 / 18) : 89%|████████▊ | 39.4GB / 44.5GB, 1.52GB/s New Data Upload : 73%|███████▎ | 3.93GB / 5.36GB, 215MB/s  ...eckpoints/model_085000.pt: 36%|███▌ | 835MB / 2.35GB  ...nts/optim_085000_rank4.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_085000_rank1.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_085000_rank3.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_085000_rank7.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_085000_rank2.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_085000_rank5.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_085000_rank6.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 82%|████████▏ | 2.03GB / 2.49GB  Processing Files (9 / 18) : 89%|████████▉ | 39.5GB / 44.5GB, 1.51GB/s New Data Upload : 74%|███████▍ | 3.97GB / 5.36GB, 216MB/s  ...eckpoints/model_085000.pt: 37%|███▋ | 870MB / 2.35GB  ...nts/optim_085000_rank4.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_085000_rank1.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_085000_rank3.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_085000_rank7.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_085000_rank2.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_085000_rank5.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_085000_rank6.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 82%|████████▏ | 2.04GB / 2.49GB  Processing Files (9 / 18) : 89%|████████▉ | 39.6GB / 44.5GB, 1.50GB/s New Data Upload : 75%|███████▌ | 4.02GB / 5.36GB, 217MB/s  ...eckpoints/model_085000.pt: 38%|███▊ | 897MB / 2.35GB  ...nts/optim_085000_rank4.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_085000_rank1.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_085000_rank3.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_085000_rank7.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_085000_rank2.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_085000_rank5.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_085000_rank6.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 83%|████████▎ | 2.06GB / 2.49GB  Processing Files (9 / 18) : 89%|████████▉ | 39.7GB / 44.5GB, 1.50GB/s New Data Upload : 75%|███████▌ | 4.07GB / 5.43GB, 218MB/s  ...eckpoints/model_085000.pt: 40%|███▉ | 932MB / 2.35GB  ...nts/optim_085000_rank4.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_085000_rank1.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_085000_rank3.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_085000_rank7.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_085000_rank2.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_085000_rank5.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_085000_rank6.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 83%|████████▎ | 2.07GB / 2.49GB  Processing Files (9 / 18) : 90%|████████▉ | 39.9GB / 44.5GB, 1.50GB/s New Data Upload : 76%|███████▌ | 4.13GB / 5.43GB, 220MB/s  ...eckpoints/model_085000.pt: 41%|████ | 959MB / 2.35GB  ...nts/optim_085000_rank4.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_085000_rank1.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_085000_rank3.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_085000_rank7.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_085000_rank2.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_085000_rank5.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_085000_rank6.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 84%|████████▎ | 2.08GB / 2.49GB  Processing Files (9 / 18) : 90%|████████▉ | 40.0GB / 44.5GB, 1.49GB/s New Data Upload : 76%|███████▌ | 4.17GB / 5.50GB, 221MB/s  ...eckpoints/model_085000.pt: 42%|████▏ | 982MB / 2.35GB  ...nts/optim_085000_rank4.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_085000_rank1.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_085000_rank3.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_085000_rank7.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_085000_rank2.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_085000_rank5.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_085000_rank6.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 84%|████████▍ | 2.09GB / 2.49GB  Processing Files (9 / 18) : 90%|█████████ | 40.1GB / 44.5GB, 1.49GB/s New Data Upload : 76%|███████▌ | 4.22GB / 5.56GB, 222MB/s  ...eckpoints/model_085000.pt: 43%|████▎ | 1.02GB / 2.35GB  ...nts/optim_085000_rank4.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_085000_rank1.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_085000_rank3.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_085000_rank7.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_085000_rank2.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_085000_rank5.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_085000_rank6.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 85%|████████▍ | 2.10GB / 2.49GB  Processing Files (9 / 18) : 90%|█████████ | 40.2GB / 44.5GB, 1.48GB/s New Data Upload : 76%|███████▌ | 4.27GB / 5.63GB, 223MB/s  ...eckpoints/model_085000.pt: 45%|████▍ | 1.05GB / 2.35GB  ...nts/optim_085000_rank4.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_085000_rank1.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_085000_rank3.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_085000_rank7.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_085000_rank2.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_085000_rank5.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_085000_rank6.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 85%|████████▌ | 2.11GB / 2.49GB  Processing Files (9 / 18) : 91%|█████████ | 40.3GB / 44.5GB, 1.48GB/s New Data Upload : 77%|███████▋ | 4.32GB / 5.63GB, 226MB/s  ...eckpoints/model_085000.pt: 46%|████▌ | 1.08GB / 2.35GB  ...nts/optim_085000_rank4.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_085000_rank1.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_085000_rank3.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_085000_rank7.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_085000_rank2.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_085000_rank5.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_085000_rank6.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 86%|████████▌ | 2.13GB / 2.49GB  Processing Files (9 / 18) : 91%|█████████ | 40.4GB / 44.5GB, 1.48GB/s New Data Upload : 77%|███████▋ | 4.37GB / 5.70GB, 226MB/s  ...eckpoints/model_085000.pt: 47%|████▋ | 1.11GB / 2.35GB  ...nts/optim_085000_rank4.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_085000_rank1.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_085000_rank3.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_085000_rank7.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_085000_rank2.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_085000_rank5.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_085000_rank6.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 86%|████████▌ | 2.14GB / 2.49GB  Processing Files (9 / 18) : 91%|█████████ | 40.6GB / 44.5GB, 1.47GB/s New Data Upload : 78%|███████▊ | 4.42GB / 5.70GB, 227MB/s  ...eckpoints/model_085000.pt: 48%|████▊ | 1.13GB / 2.35GB  ...nts/optim_085000_rank4.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_085000_rank1.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_085000_rank3.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_085000_rank7.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_085000_rank2.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_085000_rank5.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_085000_rank6.pt: 86%|████████▋ | 2.14GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 86%|████████▋ | 2.15GB / 2.49GB  Processing Files (9 / 18) : 91%|█████████▏| 40.6GB / 44.5GB, 1.46GB/s New Data Upload : 78%|███████▊ | 4.46GB / 5.70GB, 227MB/s  ...eckpoints/model_085000.pt: 49%|████▉ | 1.16GB / 2.35GB  ...nts/optim_085000_rank4.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_085000_rank1.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_085000_rank3.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_085000_rank7.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_085000_rank2.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_085000_rank5.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_085000_rank6.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 87%|████████▋ | 2.16GB / 2.49GB  Processing Files (9 / 18) : 92%|█████████▏| 40.8GB / 44.5GB, 1.46GB/s New Data Upload : 78%|███████▊ | 4.51GB / 5.76GB, 228MB/s  ...eckpoints/model_085000.pt: 51%|█████ | 1.19GB / 2.35GB  ...nts/optim_085000_rank4.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_085000_rank1.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_085000_rank3.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_085000_rank7.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_085000_rank2.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_085000_rank5.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_085000_rank6.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 87%|████████▋ | 2.17GB / 2.49GB  Processing Files (9 / 18) : 92%|█████████▏| 40.9GB / 44.5GB, 1.44GB/s New Data Upload : 79%|███████▉ | 4.55GB / 5.76GB, 228MB/s  ...eckpoints/model_085000.pt: 52%|█████▏ | 1.22GB / 2.35GB  ...nts/optim_085000_rank4.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_085000_rank1.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_085000_rank3.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_085000_rank7.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_085000_rank2.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_085000_rank5.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_085000_rank6.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 88%|████████▊ | 2.18GB / 2.49GB  Processing Files (9 / 18) : 92%|█████████▏| 41.0GB / 44.5GB, 1.44GB/s New Data Upload : 79%|███████▉ | 4.59GB / 5.83GB, 228MB/s  ...eckpoints/model_085000.pt: 53%|█████▎ | 1.25GB / 2.35GB  ...nts/optim_085000_rank4.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_085000_rank1.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_085000_rank3.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_085000_rank7.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_085000_rank2.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_085000_rank5.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_085000_rank6.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 88%|████████▊ | 2.19GB / 2.49GB  Processing Files (9 / 18) : 92%|█████████▏| 41.1GB / 44.5GB, 1.43GB/s New Data Upload : 79%|███████▊ | 4.64GB / 5.90GB, 229MB/s  ...eckpoints/model_085000.pt: 54%|█████▍ | 1.27GB / 2.35GB  ...nts/optim_085000_rank4.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_085000_rank1.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_085000_rank3.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_085000_rank7.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_085000_rank2.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_085000_rank5.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_085000_rank6.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 89%|████████▊ | 2.20GB / 2.49GB  Processing Files (9 / 18) : 93%|█████████▎| 41.2GB / 44.5GB, 1.42GB/s New Data Upload : 79%|███████▉ | 4.68GB / 5.90GB, 229MB/s  ...eckpoints/model_085000.pt: 55%|█████▌ | 1.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_085000_rank1.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_085000_rank3.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_085000_rank7.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_085000_rank2.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_085000_rank5.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_085000_rank6.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 89%|████████▉ | 2.22GB / 2.49GB  Processing Files (9 / 18) : 93%|█████████▎| 41.3GB / 44.5GB, 1.42GB/s New Data Upload : 79%|███████▉ | 4.73GB / 5.97GB, 230MB/s  ...eckpoints/model_085000.pt: 57%|█████▋ | 1.34GB / 2.35GB  ...nts/optim_085000_rank4.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_085000_rank1.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_085000_rank3.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_085000_rank7.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_085000_rank2.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_085000_rank5.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_085000_rank6.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 89%|████████▉ | 2.22GB / 2.49GB  Processing Files (9 / 18) : 93%|█████████▎| 41.4GB / 44.5GB, 1.41GB/s New Data Upload : 80%|████████ | 4.78GB / 5.97GB, 230MB/s  ...eckpoints/model_085000.pt: 58%|█████▊ | 1.37GB / 2.35GB  ...nts/optim_085000_rank4.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_085000_rank1.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_085000_rank3.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_085000_rank7.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_085000_rank2.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_085000_rank5.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_085000_rank6.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 90%|████████▉ | 2.23GB / 2.49GB  Processing Files (9 / 18) : 93%|█████████▎| 41.5GB / 44.5GB, 1.40GB/s New Data Upload : 80%|███████▉ | 4.82GB / 6.03GB, 230MB/s  ...eckpoints/model_085000.pt: 60%|█████▉ | 1.40GB / 2.35GB  ...nts/optim_085000_rank4.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_085000_rank1.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_085000_rank3.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_085000_rank7.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_085000_rank2.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_085000_rank5.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_085000_rank6.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 90%|█████████ | 2.24GB / 2.49GB  Processing Files (9 / 18) : 94%|█████████▎| 41.6GB / 44.5GB, 1.39GB/s New Data Upload : 80%|███████▉ | 4.87GB / 6.10GB, 231MB/s  ...eckpoints/model_085000.pt: 61%|██████ | 1.44GB / 2.35GB  ...nts/optim_085000_rank4.pt: 90%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_085000_rank1.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_085000_rank3.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_085000_rank7.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_085000_rank2.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_085000_rank5.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_085000_rank6.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 90%|█████████ | 2.25GB / 2.49GB  Processing Files (9 / 18) : 94%|█████████▍| 41.7GB / 44.5GB, 1.38GB/s New Data Upload : 81%|████████ | 4.92GB / 6.10GB, 230MB/s  ...eckpoints/model_085000.pt: 63%|██████▎ | 1.47GB / 2.35GB  ...nts/optim_085000_rank4.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_085000_rank1.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_085000_rank3.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_085000_rank7.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_085000_rank2.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_085000_rank5.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_085000_rank6.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 91%|█████████ | 2.26GB / 2.49GB  Processing Files (9 / 18) : 94%|█████████▍| 41.8GB / 44.5GB, 1.37GB/s New Data Upload : 81%|████████ | 4.97GB / 6.17GB, 231MB/s  ...eckpoints/model_085000.pt: 64%|██████▍ | 1.51GB / 2.35GB  ...nts/optim_085000_rank4.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_085000_rank1.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_085000_rank3.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_085000_rank7.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_085000_rank2.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_085000_rank5.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_085000_rank6.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 91%|█████████ | 2.27GB / 2.49GB  Processing Files (9 / 18) : 94%|█████████▍| 41.9GB / 44.5GB, 1.36GB/s New Data Upload : 81%|████████ | 5.02GB / 6.23GB, 231MB/s  ...eckpoints/model_085000.pt: 66%|██████▌ | 1.54GB / 2.35GB  ...nts/optim_085000_rank4.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_085000_rank1.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_085000_rank3.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_085000_rank7.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_085000_rank2.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_085000_rank5.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_085000_rank6.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 91%|█████████▏| 2.27GB / 2.49GB  Processing Files (9 / 18) : 94%|█████████▍| 42.0GB / 44.5GB, 1.35GB/s New Data Upload : 81%|████████ | 5.07GB / 6.26GB, 231MB/s  ...eckpoints/model_085000.pt: 67%|██████▋ | 1.57GB / 2.35GB  ...nts/optim_085000_rank4.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_085000_rank1.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_085000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_085000_rank7.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_085000_rank2.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_085000_rank5.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_085000_rank6.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 92%|█████████▏| 2.28GB / 2.49GB  Processing Files (9 / 18) : 95%|█████████▍| 42.1GB / 44.5GB, 1.33GB/s New Data Upload : 82%|████████▏ | 5.11GB / 6.26GB, 231MB/s  ...eckpoints/model_085000.pt: 69%|██████▊ | 1.61GB / 2.35GB  ...nts/optim_085000_rank4.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_085000_rank1.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_085000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_085000_rank7.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_085000_rank2.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_085000_rank5.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_085000_rank6.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 92%|█████████▏| 2.29GB / 2.49GB  Processing Files (9 / 18) : 95%|█████████▍| 42.2GB / 44.5GB, 1.32GB/s New Data Upload : 82%|████████▏ | 5.16GB / 6.26GB, 230MB/s  ...eckpoints/model_085000.pt: 70%|██████▉ | 1.64GB / 2.35GB  ...nts/optim_085000_rank4.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_085000_rank1.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_085000_rank3.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_085000_rank7.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_085000_rank2.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_085000_rank5.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_085000_rank6.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 92%|█████████▏| 2.30GB / 2.49GB  Processing Files (9 / 18) : 95%|█████████▌| 42.3GB / 44.5GB, 1.31GB/s New Data Upload : 83%|████████▎ | 5.21GB / 6.26GB, 230MB/s  ...eckpoints/model_085000.pt: 72%|███████▏ | 1.69GB / 2.35GB  ...nts/optim_085000_rank4.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_085000_rank1.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_085000_rank3.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_085000_rank7.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_085000_rank2.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_085000_rank5.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_085000_rank6.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 93%|█████████▎| 2.31GB / 2.49GB  Processing Files (9 / 18) : 95%|█████████▌| 42.4GB / 44.5GB, 1.31GB/s New Data Upload : 84%|████████▍ | 5.27GB / 6.26GB, 233MB/s  ...eckpoints/model_085000.pt: 73%|███████▎ | 1.71GB / 2.35GB  ...nts/optim_085000_rank4.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_085000_rank1.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_085000_rank3.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_085000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_085000_rank2.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_085000_rank5.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_085000_rank6.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 93%|█████████▎| 2.31GB / 2.49GB  Processing Files (9 / 18) : 95%|█████████▌| 42.5GB / 44.5GB, 1.28GB/s New Data Upload : 85%|████████▍ | 5.30GB / 6.26GB, 231MB/s  ...eckpoints/model_085000.pt: 74%|███████▍ | 1.75GB / 2.35GB  ...nts/optim_085000_rank4.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_085000_rank1.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_085000_rank3.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_085000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_085000_rank2.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_085000_rank5.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_085000_rank6.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 93%|█████████▎| 2.32GB / 2.49GB  Processing Files (9 / 18) : 96%|█████████▌| 42.6GB / 44.5GB, 1.25GB/s New Data Upload : 86%|████████▌ | 5.35GB / 6.26GB, 232MB/s  ...eckpoints/model_085000.pt: 75%|███████▌ | 1.77GB / 2.35GB  ...nts/optim_085000_rank4.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_085000_rank1.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_085000_rank3.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_085000_rank7.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_085000_rank2.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_085000_rank5.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_085000_rank6.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 94%|█████████▎| 2.33GB / 2.49GB  Processing Files (9 / 18) : 96%|█████████▌| 42.6GB / 44.5GB, 1.22GB/s New Data Upload : 86%|████████▌ | 5.38GB / 6.26GB, 231MB/s  ...eckpoints/model_085000.pt: 76%|███████▋ | 1.79GB / 2.35GB  ...nts/optim_085000_rank4.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_085000_rank1.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_085000_rank3.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_085000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_085000_rank2.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_085000_rank5.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_085000_rank6.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  Processing Files (9 / 18) : 96%|█████████▌| 42.7GB / 44.5GB, 1.18GB/s New Data Upload : 87%|████████▋ | 5.43GB / 6.26GB, 230MB/s  ...eckpoints/model_085000.pt: 77%|███████▋ | 1.81GB / 2.35GB  ...nts/optim_085000_rank4.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_085000_rank1.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_085000_rank3.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_085000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_085000_rank2.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_085000_rank5.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_085000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  Processing Files (9 / 18) : 96%|█████████▌| 42.8GB / 44.5GB, 1.14GB/s New Data Upload : 87%|████████▋ | 5.46GB / 6.26GB, 228MB/s  ...eckpoints/model_085000.pt: 78%|███████▊ | 1.83GB / 2.35GB  ...nts/optim_085000_rank4.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_085000_rank1.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_085000_rank3.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_085000_rank7.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_085000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank5.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_085000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 94%|█████████▍| 2.35GB / 2.49GB  Processing Files (9 / 18) : 96%|█████████▋| 42.8GB / 44.5GB, 1.09GB/s New Data Upload : 88%|████████▊ | 5.49GB / 6.26GB, 226MB/s  ...eckpoints/model_085000.pt: 79%|███████▉ | 1.86GB / 2.35GB  ...nts/optim_085000_rank4.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_085000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_085000_rank3.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_085000_rank7.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_085000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank5.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_085000_rank6.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 95%|█████████▍| 2.35GB / 2.49GB  Processing Files (9 / 18) : 96%|█████████▋| 42.9GB / 44.5GB, 1.04GB/s New Data Upload : 88%|████████▊ | 5.52GB / 6.26GB, 227MB/s  ...eckpoints/model_085000.pt: 80%|███████▉ | 1.88GB / 2.35GB  ...nts/optim_085000_rank4.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_085000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_085000_rank3.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_085000_rank7.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_085000_rank2.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank5.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_085000_rank6.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 95%|█████████▍| 2.36GB / 2.49GB  Processing Files (9 / 18) : 97%|█████████▋| 42.9GB / 44.5GB, 997MB/s New Data Upload : 89%|████████▊ | 5.55GB / 6.26GB, 225MB/s  ...eckpoints/model_085000.pt: 81%|████████ | 1.90GB / 2.35GB  ...nts/optim_085000_rank4.pt: 94%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_085000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_085000_rank3.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_085000_rank7.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_085000_rank2.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank6.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 95%|█████████▌| 2.36GB / 2.49GB  Processing Files (9 / 18) : 97%|█████████▋| 43.0GB / 44.5GB, 959MB/s New Data Upload : 89%|████████▉ | 5.59GB / 6.26GB, 225MB/s  ...eckpoints/model_085000.pt: 82%|████████▏ | 1.92GB / 2.35GB  ...nts/optim_085000_rank4.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_085000_rank1.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_085000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_085000_rank7.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_085000_rank2.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_085000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank6.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  Processing Files (9 / 18) : 97%|█████████▋| 43.1GB / 44.5GB, 913MB/s New Data Upload : 90%|████████▉ | 5.61GB / 6.26GB, 222MB/s  ...eckpoints/model_085000.pt: 82%|████████▏ | 1.94GB / 2.35GB  ...nts/optim_085000_rank4.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_085000_rank1.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_085000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_085000_rank7.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_085000_rank2.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank6.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  Processing Files (9 / 18) : 97%|█████████▋| 43.1GB / 44.5GB, 875MB/s New Data Upload : 90%|█████████ | 5.64GB / 6.26GB, 222MB/s  ...eckpoints/model_085000.pt: 83%|████████▎ | 1.95GB / 2.35GB  ...nts/optim_085000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_085000_rank1.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_085000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank7.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_085000_rank2.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank5.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank6.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (9 / 18) : 97%|█████████▋| 43.2GB / 44.5GB, 830MB/s New Data Upload : 90%|█████████ | 5.66GB / 6.26GB, 220MB/s  ...eckpoints/model_085000.pt: 84%|████████▍ | 1.97GB / 2.35GB  ...nts/optim_085000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_085000_rank1.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_085000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank5.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank6.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (9 / 18) : 97%|█████████▋| 43.2GB / 44.5GB, 788MB/s New Data Upload : 91%|█████████ | 5.69GB / 6.26GB, 219MB/s  ...eckpoints/model_085000.pt: 84%|████████▍ | 1.98GB / 2.35GB  ...nts/optim_085000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_085000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_085000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank5.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (9 / 18) : 97%|█████████▋| 43.2GB / 44.5GB, 744MB/s New Data Upload : 91%|█████████ | 5.71GB / 6.26GB, 216MB/s  ...eckpoints/model_085000.pt: 85%|████████▌ | 2.00GB / 2.35GB  ...nts/optim_085000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_085000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_085000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank5.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_085000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (9 / 18) : 97%|█████████▋| 43.3GB / 44.5GB, 697MB/s New Data Upload : 91%|█████████▏| 5.72GB / 6.26GB, 214MB/s  ...eckpoints/model_085000.pt: 85%|████████▌ | 2.01GB / 2.35GB  ...nts/optim_085000_rank4.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_085000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_085000_rank3.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank5.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 96%|█████████▌| 2.39GB / 2.49GB  Processing Files (9 / 18) : 97%|█████████▋| 43.3GB / 44.5GB, 656MB/s New Data Upload : 92%|█████████▏| 5.74GB / 6.26GB, 211MB/s  ...eckpoints/model_085000.pt: 86%|████████▌ | 2.02GB / 2.35GB  ...nts/optim_085000_rank4.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_085000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_085000_rank3.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank7.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank5.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 96%|█████████▌| 2.39GB / 2.49GB  Processing Files (9 / 18) : 97%|█████████▋| 43.3GB / 44.5GB, 606MB/s New Data Upload : 92%|█████████▏| 5.76GB / 6.26GB, 209MB/s  ...eckpoints/model_085000.pt: 87%|████████▋ | 2.03GB / 2.35GB  ...nts/optim_085000_rank4.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_085000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank3.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_085000_rank7.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 96%|█████████▋| 2.39GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.4GB / 44.5GB, 542MB/s New Data Upload : 92%|█████████▏| 5.78GB / 6.26GB, 206MB/s  ...eckpoints/model_085000.pt: 87%|████████▋ | 2.05GB / 2.35GB  ...nts/optim_085000_rank4.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_085000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank3.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_085000_rank7.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 96%|█████████▋| 2.39GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.4GB / 44.5GB, 481MB/s New Data Upload : 93%|█████████▎| 5.79GB / 6.26GB, 203MB/s  ...eckpoints/model_085000.pt: 88%|████████▊ | 2.06GB / 2.35GB  ...nts/optim_085000_rank4.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_085000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank3.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank7.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 96%|█████████▋| 2.40GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.4GB / 44.5GB, 460MB/s New Data Upload : 93%|█████████▎| 5.81GB / 6.26GB, 200MB/s  ...eckpoints/model_085000.pt: 88%|████████▊ | 2.07GB / 2.35GB  ...nts/optim_085000_rank4.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_085000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank3.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank7.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_085000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 96%|█████████▋| 2.40GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.5GB / 44.5GB, 436MB/s New Data Upload : 93%|█████████▎| 5.82GB / 6.26GB, 197MB/s  ...eckpoints/model_085000.pt: 89%|████████▊ | 2.08GB / 2.35GB  ...nts/optim_085000_rank4.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_085000_rank1.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank7.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank6.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.40GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.5GB / 44.5GB, 417MB/s New Data Upload : 93%|█████████▎| 5.84GB / 6.26GB, 192MB/s  ...eckpoints/model_085000.pt: 89%|████████▉ | 2.10GB / 2.35GB  ...nts/optim_085000_rank4.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_085000_rank1.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank6.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.40GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.5GB / 44.5GB, 407MB/s New Data Upload : 94%|█████████▎| 5.86GB / 6.26GB, 189MB/s  ...eckpoints/model_085000.pt: 90%|████████▉ | 2.11GB / 2.35GB  ...nts/optim_085000_rank4.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_085000_rank1.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank6.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.40GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.5GB / 44.5GB, 396MB/s New Data Upload : 94%|█████████▍| 5.87GB / 6.26GB, 186MB/s  ...eckpoints/model_085000.pt: 90%|█████████ | 2.12GB / 2.35GB  ...nts/optim_085000_rank4.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_085000_rank1.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank6.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.6GB / 44.5GB, 388MB/s New Data Upload : 94%|█████████▍| 5.89GB / 6.26GB, 183MB/s  ...eckpoints/model_085000.pt: 91%|█████████ | 2.13GB / 2.35GB  ...nts/optim_085000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank1.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.6GB / 44.5GB, 380MB/s New Data Upload : 94%|█████████▍| 5.91GB / 6.26GB, 180MB/s  ...eckpoints/model_085000.pt: 91%|█████████ | 2.14GB / 2.35GB  ...nts/optim_085000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank1.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.6GB / 44.5GB, 369MB/s New Data Upload : 95%|█████████▍| 5.92GB / 6.26GB, 176MB/s  ...eckpoints/model_085000.pt: 92%|█████████▏| 2.15GB / 2.35GB  ...nts/optim_085000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.7GB / 44.5GB, 360MB/s New Data Upload : 95%|█████████▍| 5.93GB / 6.26GB, 173MB/s  ...eckpoints/model_085000.pt: 92%|█████████▏| 2.16GB / 2.35GB  ...nts/optim_085000_rank4.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.7GB / 44.5GB, 349MB/s New Data Upload : 95%|█████████▍| 5.94GB / 6.26GB, 169MB/s  ...eckpoints/model_085000.pt: 92%|█████████▏| 2.17GB / 2.35GB  ...nts/optim_085000_rank4.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.7GB / 44.5GB, 341MB/s New Data Upload : 95%|█████████▌| 5.96GB / 6.26GB, 165MB/s  ...eckpoints/model_085000.pt: 92%|█████████▏| 2.17GB / 2.35GB  ...nts/optim_085000_rank4.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.7GB / 44.5GB, 331MB/s New Data Upload : 95%|█████████▌| 5.97GB / 6.26GB, 161MB/s  ...eckpoints/model_085000.pt: 93%|█████████▎| 2.18GB / 2.35GB  ...nts/optim_085000_rank4.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.7GB / 44.5GB, 322MB/s New Data Upload : 96%|█████████▌| 5.98GB / 6.26GB, 158MB/s  ...eckpoints/model_085000.pt: 93%|█████████▎| 2.19GB / 2.35GB  ...nts/optim_085000_rank4.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.7GB / 44.5GB, 312MB/s New Data Upload : 96%|█████████▌| 5.99GB / 6.26GB, 154MB/s  ...eckpoints/model_085000.pt: 93%|█████████▎| 2.20GB / 2.35GB  ...nts/optim_085000_rank4.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.8GB / 44.5GB, 305MB/s New Data Upload : 96%|█████████▌| 6.00GB / 6.26GB, 151MB/s  ...eckpoints/model_085000.pt: 94%|█████████▍| 2.20GB / 2.35GB  ...nts/optim_085000_rank4.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.8GB / 44.5GB, 294MB/s New Data Upload : 96%|█████████▌| 6.00GB / 6.26GB, 147MB/s  ...eckpoints/model_085000.pt: 94%|█████████▍| 2.21GB / 2.35GB  ...nts/optim_085000_rank4.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.8GB / 44.5GB, 285MB/s New Data Upload : 96%|█████████▌| 6.02GB / 6.26GB, 144MB/s  ...eckpoints/model_085000.pt: 94%|█████████▍| 2.22GB / 2.35GB  ...nts/optim_085000_rank4.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (9 / 18) : 98%|█████████▊| 43.8GB / 44.5GB, 275MB/s New Data Upload : 96%|█████████▌| 6.02GB / 6.26GB, 140MB/s  ...eckpoints/model_085000.pt: 95%|█████████▍| 2.22GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▊| 43.8GB / 44.5GB, 265MB/s New Data Upload : 96%|█████████▋| 6.03GB / 6.26GB, 136MB/s  ...eckpoints/model_085000.pt: 95%|█████████▍| 2.23GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▊| 43.8GB / 44.5GB, 257MB/s New Data Upload : 96%|█████████▋| 6.04GB / 6.26GB, 133MB/s  ...eckpoints/model_085000.pt: 95%|█████████▌| 2.24GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.42GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▊| 43.8GB / 44.5GB, 245MB/s New Data Upload : 97%|█████████▋| 6.04GB / 6.26GB, 129MB/s  ...eckpoints/model_085000.pt: 95%|█████████▌| 2.24GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▊| 43.9GB / 44.5GB, 239MB/s New Data Upload : 97%|█████████▋| 6.05GB / 6.26GB, 125MB/s  ...eckpoints/model_085000.pt: 95%|█████████▌| 2.24GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▊| 43.9GB / 44.5GB, 232MB/s New Data Upload : 97%|█████████▋| 6.05GB / 6.26GB, 121MB/s  ...eckpoints/model_085000.pt: 96%|█████████▌| 2.25GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▊| 43.9GB / 44.5GB, 219MB/s New Data Upload : 97%|█████████▋| 6.06GB / 6.26GB, 116MB/s  ...eckpoints/model_085000.pt: 96%|█████████▌| 2.25GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▊| 43.9GB / 44.5GB, 215MB/s New Data Upload : 97%|█████████▋| 6.07GB / 6.26GB, 113MB/s  ...eckpoints/model_085000.pt: 96%|█████████▌| 2.25GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▊| 43.9GB / 44.5GB, 206MB/s New Data Upload : 97%|█████████▋| 6.07GB / 6.26GB, 108MB/s  ...eckpoints/model_085000.pt: 96%|█████████▌| 2.26GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▊| 43.9GB / 44.5GB, 196MB/s New Data Upload : 97%|█████████▋| 6.07GB / 6.26GB, 103MB/s  ...eckpoints/model_085000.pt: 96%|█████████▋| 2.26GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 43.9GB / 44.5GB, 189MB/s New Data Upload : 97%|█████████▋| 6.08GB / 6.26GB, 99.4MB/s  ...eckpoints/model_085000.pt: 96%|█████████▋| 2.26GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 43.9GB / 44.5GB, 180MB/s New Data Upload : 97%|█████████▋| 6.08GB / 6.26GB, 95.2MB/s  ...eckpoints/model_085000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 43.9GB / 44.5GB, 173MB/s New Data Upload : 97%|█████████▋| 6.09GB / 6.26GB, 91.0MB/s  ...eckpoints/model_085000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 43.9GB / 44.5GB, 165MB/s New Data Upload : 97%|█████████▋| 6.09GB / 6.26GB, 87.0MB/s  ...eckpoints/model_085000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.0GB / 44.5GB, 153MB/s New Data Upload : 97%|█████████▋| 6.10GB / 6.26GB, 81.0MB/s  ...eckpoints/model_085000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.0GB / 44.5GB, 148MB/s New Data Upload : 97%|█████████▋| 6.10GB / 6.26GB, 78.3MB/s  ...eckpoints/model_085000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.0GB / 44.5GB, 140MB/s New Data Upload : 98%|█████████▊| 6.11GB / 6.26GB, 73.6MB/s  ...eckpoints/model_085000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.0GB / 44.5GB, 135MB/s New Data Upload : 98%|█████████▊| 6.11GB / 6.26GB, 71.2MB/s  ...eckpoints/model_085000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.0GB / 44.5GB, 128MB/s New Data Upload : 98%|█████████▊| 6.11GB / 6.26GB, 67.4MB/s  ...eckpoints/model_085000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.0GB / 44.5GB, 122MB/s New Data Upload : 98%|█████████▊| 6.12GB / 6.26GB, 64.6MB/s  ...eckpoints/model_085000.pt: 97%|█████████▋| 2.29GB / 2.35GB  ...nts/optim_085000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.0GB / 44.5GB, 118MB/s New Data Upload : 98%|█████████▊| 6.12GB / 6.26GB, 62.0MB/s  ...eckpoints/model_085000.pt: 97%|█████████▋| 2.29GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.0GB / 44.5GB, 112MB/s New Data Upload : 98%|█████████▊| 6.12GB / 6.26GB, 58.7MB/s  ...eckpoints/model_085000.pt: 97%|█████████▋| 2.29GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.1GB / 44.5GB, 110MB/s New Data Upload : 98%|█████████▊| 6.13GB / 6.26GB, 56.9MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.29GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.1GB / 44.5GB, 104MB/s New Data Upload : 98%|█████████▊| 6.13GB / 6.26GB, 53.3MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.29GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.1GB / 44.5GB, 100MB/s New Data Upload : 98%|█████████▊| 6.13GB / 6.26GB, 51.6MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.29GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.1GB / 44.5GB, 96.8MB/s New Data Upload : 98%|█████████▊| 6.14GB / 6.26GB, 49.1MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.1GB / 44.5GB, 91.3MB/s New Data Upload : 98%|█████████▊| 6.14GB / 6.26GB, 46.8MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.1GB / 44.5GB, 88.4MB/s New Data Upload : 98%|█████████▊| 6.14GB / 6.26GB, 44.6MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.1GB / 44.5GB, 85.0MB/s New Data Upload : 98%|█████████▊| 6.14GB / 6.26GB, 43.0MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.1GB / 44.5GB, 83.4MB/s New Data Upload : 98%|█████████▊| 6.15GB / 6.26GB, 41.6MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.1GB / 44.5GB, 80.4MB/s New Data Upload : 98%|█████████▊| 6.15GB / 6.26GB, 40.4MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank1.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank7.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank6.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.1GB / 44.5GB, 74.4MB/s New Data Upload : 98%|█████████▊| 6.15GB / 6.26GB, 36.8MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.2GB / 44.5GB, 73.7MB/s New Data Upload : 98%|█████████▊| 6.15GB / 6.26GB, 35.6MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.2GB / 44.5GB, 71.9MB/s New Data Upload : 98%|█████████▊| 6.16GB / 6.26GB, 34.0MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.2GB / 44.5GB, 70.5MB/s New Data Upload : 98%|█████████▊| 6.16GB / 6.26GB, 33.0MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.2GB / 44.5GB, 66.0MB/s New Data Upload : 98%|█████████▊| 6.16GB / 6.26GB, 31.0MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.2GB / 44.5GB, 64.6MB/s New Data Upload : 98%|█████████▊| 6.16GB / 6.26GB, 29.7MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.2GB / 44.5GB, 64.1MB/s New Data Upload : 99%|█████████▊| 6.17GB / 6.26GB, 28.7MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.2GB / 44.5GB, 61.9MB/s New Data Upload : 99%|█████████▊| 6.17GB / 6.26GB, 26.8MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.2GB / 44.5GB, 57.4MB/s New Data Upload : 99%|█████████▊| 6.17GB / 6.26GB, 24.2MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.2GB / 44.5GB, 57.2MB/s New Data Upload : 99%|█████████▊| 6.17GB / 6.26GB, 23.5MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (9 / 18) : 99%|█████████▉| 44.2GB / 44.5GB, 57.0MB/s New Data Upload : 99%|█████████▊| 6.18GB / 6.26GB, 22.8MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.2GB / 44.5GB, 55.7MB/s New Data Upload : 99%|█████████▊| 6.18GB / 6.26GB, 21.7MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 54.5MB/s New Data Upload : 99%|█████████▊| 6.18GB / 6.26GB, 21.0MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 53.0MB/s New Data Upload : 99%|█████████▉| 6.18GB / 6.26GB, 19.8MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 52.1MB/s New Data Upload : 99%|█████████▉| 6.18GB / 6.26GB, 19.2MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 51.5MB/s New Data Upload : 99%|█████████▉| 6.19GB / 6.26GB, 18.8MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 50.9MB/s New Data Upload : 99%|█████████▉| 6.19GB / 6.26GB, 18.1MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 50.5MB/s New Data Upload : 99%|█████████▉| 6.19GB / 6.26GB, 17.2MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 51.4MB/s New Data Upload : 99%|█████████▉| 6.19GB / 6.26GB, 17.2MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 50.3MB/s New Data Upload : 99%|█████████▉| 6.20GB / 6.26GB, 16.6MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 51.0MB/s New Data Upload : 99%|█████████▉| 6.20GB / 6.26GB, 16.2MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 50.5MB/s New Data Upload : 99%|█████████▉| 6.20GB / 6.26GB, 15.6MB/s  ...eckpoints/model_085000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.3GB / 44.5GB, 48.4MB/s New Data Upload : 99%|█████████▉| 6.20GB / 6.26GB, 14.8MB/s  ...eckpoints/model_085000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 49.1MB/s New Data Upload : 99%|█████████▉| 6.21GB / 6.26GB, 14.9MB/s  ...eckpoints/model_085000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 49.9MB/s New Data Upload : 99%|█████████▉| 6.21GB / 6.26GB, 14.8MB/s  ...eckpoints/model_085000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 48.9MB/s New Data Upload : 99%|█████████▉| 6.21GB / 6.26GB, 14.4MB/s  ...eckpoints/model_085000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_085000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 49.2MB/s New Data Upload : 99%|█████████▉| 6.22GB / 6.26GB, 14.3MB/s  ...eckpoints/model_085000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 49.8MB/s New Data Upload : 99%|█████████▉| 6.22GB / 6.26GB, 14.3MB/s  ...eckpoints/model_085000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 49.1MB/s New Data Upload : 99%|█████████▉| 6.22GB / 6.26GB, 13.9MB/s  ...eckpoints/model_085000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 49.3MB/s New Data Upload : 99%|█████████▉| 6.23GB / 6.26GB, 13.8MB/s  ...eckpoints/model_085000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 48.4MB/s New Data Upload : 99%|█████████▉| 6.23GB / 6.26GB, 13.6MB/s  ...eckpoints/model_085000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 48.2MB/s New Data Upload : 100%|█████████▉| 6.23GB / 6.26GB, 13.6MB/s  ...eckpoints/model_085000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 47.0MB/s New Data Upload : 100%|█████████▉| 6.23GB / 6.26GB, 13.4MB/s  ...eckpoints/model_085000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 46.5MB/s New Data Upload : 100%|█████████▉| 6.23GB / 6.26GB, 13.2MB/s  ...eckpoints/model_085000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 45.0MB/s New Data Upload : 100%|█████████▉| 6.24GB / 6.26GB, 13.0MB/s  ...eckpoints/model_085000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 44.4MB/s New Data Upload : 100%|█████████▉| 6.24GB / 6.26GB, 12.7MB/s  ...eckpoints/model_085000.pt: 99%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.4GB / 44.5GB, 43.5MB/s New Data Upload : 100%|█████████▉| 6.24GB / 6.26GB, 12.8MB/s  ...eckpoints/model_085000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.5GB / 44.5GB, 41.8MB/s New Data Upload : 100%|█████████▉| 6.24GB / 6.26GB, 12.5MB/s  ...eckpoints/model_085000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.5GB / 44.5GB, 41.5MB/s New Data Upload : 100%|█████████▉| 6.25GB / 6.26GB, 12.6MB/s  ...eckpoints/model_085000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.5GB / 44.5GB, 40.2MB/s New Data Upload : 100%|█████████▉| 6.25GB / 6.26GB, 12.5MB/s  ...eckpoints/model_085000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.5GB / 44.5GB, 39.3MB/s New Data Upload : 100%|█████████▉| 6.25GB / 6.26GB, 12.3MB/s  ...eckpoints/model_085000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.5GB / 44.5GB, 38.1MB/s New Data Upload : 100%|█████████▉| 6.26GB / 6.26GB, 12.2MB/s  ...eckpoints/model_085000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (9 / 18) : 100%|█████████▉| 44.5GB / 44.5GB, 37.7MB/s New Data Upload : 100%|█████████▉| 6.26GB / 6.26GB, 12.1MB/s  ...eckpoints/model_085000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (17 / 18) : 100%|█████████▉| 44.5GB / 44.5GB, 32.5MB/s New Data Upload : 100%|█████████▉| 6.26GB / 6.26GB, 10.5MB/s  ...eckpoints/model_085000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (18 / 18) : 100%|██████████| 44.5GB / 44.5GB, 32.5MB/s New Data Upload : 100%|██████████| 6.26GB / 6.26GB, 10.6MB/s  ...eckpoints/model_085000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_085000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_085000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_085000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (18 / 18) : 100%|██████████| 44.5GB / 44.5GB, 22.5MB/s New Data Upload : 100%|██████████| 6.26GB / 6.26GB, 8.31MB/s ...eckpoints/model_085000.pt: 100%|██████████| 2.35GB / 2.35GB ...nts/optim_085000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_085000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_085000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_085000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_085000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_085000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_085000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_075000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_085000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB [23:02:36] Step 85000 uploaded [23:57:36] Uploading step 90000... Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...nts/optim_090000_rank1.pt: 0%| | 587kB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 96.6kB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 97.3kB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 97.3kB / 2.49GB  ...nts/optim_090000_rank1.pt: 0%| | 587kB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 96.6kB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 97.3kB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 97.3kB / 2.49GB  Processing Files (0 / 4) : 0%| | 878kB / 22.2GB, 1.10MB/s New Data Upload : 0%| | 587kB / 537MB, 734kB/s  ...nts/optim_090000_rank1.pt: 0%| | 587kB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 96.6kB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 97.3kB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 97.3kB / 2.49GB  ...nts/optim_090000_rank1.pt: 0%| | 1.76MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 290kB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 292kB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 292kB / 2.49GB  Processing Files (0 / 4) : 0%| | 2.63MB / 22.2GB, 2.20MB/s New Data Upload : 0%| | 1.76MB / 604MB, 1.47MB/s  ...nts/optim_090000_rank1.pt: 0%| | 6.46MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 1.06MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 1.07MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 1.07MB / 2.49GB  Processing Files (0 / 4) : 0%| | 9.66MB / 22.2GB, 6.90MB/s New Data Upload : 1%| | 6.46MB / 604MB, 4.61MB/s  ...nts/optim_090000_rank1.pt: 0%| | 9.98MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 1.64MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 1.65MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 1.65MB / 2.49GB  Processing Files (0 / 4) : 0%| | 14.9MB / 22.2GB, 9.33MB/s New Data Upload : 2%|▏ | 9.98MB / 604MB, 6.24MB/s  ...nts/optim_090000_rank1.pt: 1%| | 15.3MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 2.51MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 2.53MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 2.53MB / 2.49GB  Processing Files (0 / 4) : 0%| | 22.8MB / 22.2GB, 12.7MB/s New Data Upload : 3%|▎ | 15.3MB / 604MB, 8.48MB/s  ...nts/optim_090000_rank1.pt: 1%| | 17.6MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 2.90MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 2.92MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 2.92MB / 2.49GB  Processing Files (0 / 4) : 0%| | 26.3MB / 22.2GB, 13.2MB/s New Data Upload : 3%|▎ | 17.6MB / 604MB, 8.81MB/s  ...nts/optim_090000_rank1.pt: 1%| | 23.5MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 3.87MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 3.89MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 3.89MB / 2.49GB  Processing Files (0 / 4) : 0%| | 35.1MB / 22.2GB, 16.0MB/s New Data Upload : 4%|▍ | 23.5MB / 604MB, 10.7MB/s  ...nts/optim_090000_rank1.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 4.45MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 4.48MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 4.48MB / 2.49GB  Processing Files (0 / 4) : 0%| | 40.4MB / 22.2GB, 16.8MB/s New Data Upload : 4%|▍ | 27.0MB / 604MB, 11.3MB/s  ...nts/optim_090000_rank1.pt: 1%|▏ | 32.3MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 5.31MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 5.35MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 5.35MB / 2.49GB  Processing Files (0 / 4) : 0%| | 48.3MB / 22.2GB, 18.6MB/s New Data Upload : 5%|▌ | 32.3MB / 604MB, 12.4MB/s  ...nts/optim_090000_rank1.pt: 1%|▏ | 36.4MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 5.99MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 6.03MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 6.03MB / 2.49GB  Processing Files (0 / 4) : 0%| | 54.4MB / 22.2GB, 19.4MB/s New Data Upload : 6%|▌ | 36.4MB / 604MB, 13.0MB/s  ...nts/optim_090000_rank1.pt: 2%|▏ | 38.7MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 6.38MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 6.42MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 6.42MB / 2.49GB  Processing Files (0 / 4) : 0%| | 58.0MB / 22.2GB, 19.3MB/s New Data Upload : 6%|▋ | 38.7MB / 604MB, 12.9MB/s  ...nts/optim_090000_rank1.pt: 2%|▏ | 42.3MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 6.96MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 7.01MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 7.01MB / 2.49GB  Processing Files (0 / 4) : 0%| | 63.2MB / 22.2GB, 19.8MB/s New Data Upload : 7%|▋ | 42.3MB / 604MB, 13.2MB/s  ...nts/optim_090000_rank1.pt: 2%|▏ | 46.4MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 7.63MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 7.69MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 7.69MB / 2.49GB  Processing Files (0 / 4) : 0%| | 69.4MB / 22.2GB, 20.4MB/s New Data Upload : 8%|▊ | 46.4MB / 604MB, 13.6MB/s  ...nts/optim_090000_rank1.pt: 2%|▏ | 49.9MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 8.21MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 8.27MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 8.27MB / 2.49GB  Processing Files (0 / 4) : 0%| | 74.6MB / 22.2GB, 20.7MB/s New Data Upload : 8%|▊ | 49.9MB / 604MB, 13.9MB/s  ...nts/optim_090000_rank1.pt: 2%|▏ | 54.0MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 8.89MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 8.95MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 8.95MB / 2.49GB  Processing Files (0 / 4) : 0%| | 80.8MB / 22.2GB, 21.3MB/s New Data Upload : 9%|▉ | 54.0MB / 604MB, 14.2MB/s  ...nts/optim_090000_rank1.pt: 2%|▏ | 57.5MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 9.47MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 9.54MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 9.54MB / 2.49GB  Processing Files (0 / 4) : 0%| | 86.1MB / 22.2GB, 21.5MB/s New Data Upload : 10%|▉ | 57.5MB / 604MB, 14.4MB/s  ...nts/optim_090000_rank1.pt: 2%|▏ | 61.0MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 10.0MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 10.1MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 10.1MB / 2.49GB  Processing Files (0 / 4) : 0%| | 91.3MB / 22.2GB, 21.7MB/s New Data Upload : 10%|█ | 61.0MB / 604MB, 14.5MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 64.6MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 10.6MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 10.7MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 10.7MB / 2.49GB  Processing Files (0 / 4) : 0%| | 96.6MB / 22.2GB, 22.0MB/s New Data Upload : 11%|█ | 64.6MB / 604MB, 14.7MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 66.9MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 11.0MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 11.1MB / 2.49GB  Processing Files (0 / 4) : 0%| | 100MB / 22.2GB, 21.8MB/s New Data Upload : 11%|█ | 66.9MB / 604MB, 14.5MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 66.9MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 11.0MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank1.pt: 3%|▎ | 66.9MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 11.0MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank1.pt: 3%|▎ | 66.9MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 11.0MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank6.pt: 0%| | 4.11MB / 2.49GB  ...nts/optim_090000_rank1.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 11.0MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank6.pt: 0%| | 4.11MB / 2.49GB  Processing Files (0 / 5) : 0%| | 104MB / 22.2GB, 19.3MB/s New Data Upload : 12%|█▏ | 71.2MB / 604MB, 13.2MB/s  ...nts/optim_090000_rank0.pt: 0%| | 2.35MB / 2.49GB  ...nts/optim_090000_rank1.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 11.0MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank6.pt: 0%| | 4.11MB / 2.49GB  ...nts/optim_090000_rank0.pt: 0%| | 2.35MB / 2.49GB  Processing Files (0 / 6) : 0%| | 107MB / 22.2GB, 19.1MB/s New Data Upload : 10%|▉ | 73.5MB / 738MB, 13.1MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 11.0MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank6.pt: 0%| | 7.04MB / 2.49GB  ...nts/optim_090000_rank0.pt: 0%| | 2.35MB / 2.49GB  Processing Files (0 / 6) : 0%| | 110MB / 22.2GB, 18.9MB/s New Data Upload : 10%|█ | 76.5MB / 738MB, 13.2MB/s  ...nts/optim_090000_rank7.pt: 0%| | 3.09MB / 2.49GB  ...nts/optim_090000_rank1.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 11.0MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank6.pt: 0%| | 11.7MB / 2.49GB  ...nts/optim_090000_rank0.pt: 0%| | 2.35MB / 2.49GB  ...nts/optim_090000_rank7.pt: 0%| | 3.09MB / 2.49GB  Processing Files (0 / 7) : 1%| | 118MB / 22.2GB, 19.6MB/s New Data Upload : 11%|█ | 81.1MB / 738MB, 13.5MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank6.pt: 1%| | 16.4MB / 2.49GB  ...nts/optim_090000_rank0.pt: 0%| | 4.70MB / 2.49GB  ...nts/optim_090000_rank7.pt: 0%| | 3.68MB / 2.49GB  Processing Files (0 / 7) : 1%| | 125MB / 22.2GB, 20.2MB/s New Data Upload : 12%|█▏ | 88.8MB / 738MB, 14.3MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_090000_rank6.pt: 1%| | 20.5MB / 2.49GB  ...nts/optim_090000_rank0.pt: 0%| | 8.22MB / 2.49GB  ...nts/optim_090000_rank7.pt: 0%| | 5.91MB / 2.49GB  Processing Files (0 / 7) : 1%| | 135MB / 22.2GB, 21.1MB/s New Data Upload : 12%|█▏ | 96.4MB / 805MB, 15.1MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 11.2MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 11.2MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 11.2MB / 2.49GB  ...nts/optim_090000_rank6.pt: 1%| | 22.3MB / 2.49GB  ...nts/optim_090000_rank0.pt: 1%| | 14.1MB / 2.49GB  ...nts/optim_090000_rank7.pt: 0%| | 6.49MB / 2.49GB  Processing Files (0 / 7) : 1%| | 144MB / 22.2GB, 21.7MB/s New Data Upload : 13%|█▎ | 105MB / 805MB, 15.9MB/s  ...eckpoints/model_090000.pt: 0%| | 5.40MB / 2.35GB  ...nts/optim_090000_rank1.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 11.9MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 11.4MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 11.4MB / 2.49GB  ...nts/optim_090000_rank6.pt: 1%| | 27.0MB / 2.49GB  ...nts/optim_090000_rank0.pt: 1%| | 20.5MB / 2.49GB  ...nts/optim_090000_rank7.pt: 0%| | 10.6MB / 2.49GB  ...eckpoints/model_090000.pt: 0%| | 5.40MB / 2.35GB  Processing Files (0 / 8) : 1%| | 165MB / 22.2GB, 24.3MB/s New Data Upload : 14%|█▍ | 122MB / 872MB, 18.0MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_090000_rank2.pt: 0%| | 12.4MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 11.5MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 11.5MB / 2.49GB  ...nts/optim_090000_rank6.pt: 1%|▏ | 31.1MB / 2.49GB  ...nts/optim_090000_rank0.pt: 1%| | 27.6MB / 2.49GB  ...nts/optim_090000_rank7.pt: 1%| | 13.5MB / 2.49GB  ...eckpoints/model_090000.pt: 0%| | 5.40MB / 2.35GB  Processing Files (0 / 8) : 1%| | 180MB / 22.2GB, 25.7MB/s New Data Upload : 16%|█▌ | 136MB / 872MB, 19.5MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_090000_rank2.pt: 1%| | 13.7MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 11.8MB / 2.49GB  ...nts/optim_090000_rank5.pt: 0%| | 11.8MB / 2.49GB  ...nts/optim_090000_rank6.pt: 1%|▏ | 35.2MB / 2.49GB  ...nts/optim_090000_rank0.pt: 1%|▏ | 32.3MB / 2.49GB  ...nts/optim_090000_rank7.pt: 1%| | 21.2MB / 2.49GB  ...eckpoints/model_090000.pt: 0%| | 5.40MB / 2.35GB  Processing Files (0 / 8) : 1%| | 198MB / 22.2GB, 27.6MB/s New Data Upload : 18%|█▊ | 153MB / 872MB, 21.2MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_090000_rank2.pt: 1%| | 14.5MB / 2.49GB  ...nts/optim_090000_rank4.pt: 0%| | 12.1MB / 2.49GB  ...nts/optim_090000_rank5.pt: 1%| | 14.9MB / 2.49GB  ...nts/optim_090000_rank6.pt: 2%|▏ | 39.3MB / 2.49GB  ...nts/optim_090000_rank0.pt: 1%|▏ | 36.4MB / 2.49GB  ...nts/optim_090000_rank7.pt: 1%| | 25.9MB / 2.49GB  ...eckpoints/model_090000.pt: 0%| | 9.45MB / 2.35GB  Processing Files (0 / 8) : 1%| | 220MB / 22.2GB, 29.7MB/s New Data Upload : 20%|█▉ | 170MB / 872MB, 23.0MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_090000_rank2.pt: 1%| | 16.0MB / 2.49GB  ...nts/optim_090000_rank4.pt: 1%| | 12.5MB / 2.49GB  ...nts/optim_090000_rank5.pt: 1%| | 15.3MB / 2.49GB  ...nts/optim_090000_rank6.pt: 2%|▏ | 41.7MB / 2.49GB  ...nts/optim_090000_rank0.pt: 2%|▏ | 41.7MB / 2.49GB  ...nts/optim_090000_rank7.pt: 1%|▏ | 35.3MB / 2.49GB  ...eckpoints/model_090000.pt: 0%| | 11.2MB / 2.35GB  Processing Files (0 / 8) : 1%| | 241MB / 22.2GB, 31.7MB/s New Data Upload : 20%|██ | 189MB / 939MB, 24.9MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_090000_rank2.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_090000_rank4.pt: 1%| | 12.7MB / 2.49GB  ...nts/optim_090000_rank5.pt: 1%| | 16.7MB / 2.49GB  ...nts/optim_090000_rank6.pt: 2%|▏ | 45.8MB / 2.49GB  ...nts/optim_090000_rank0.pt: 2%|▏ | 46.4MB / 2.49GB  ...nts/optim_090000_rank7.pt: 2%|▏ | 40.6MB / 2.49GB  ...eckpoints/model_090000.pt: 1%| | 19.3MB / 2.35GB  Processing Files (0 / 8) : 1%| | 268MB / 22.2GB, 34.3MB/s New Data Upload : 23%|██▎ | 212MB / 939MB, 27.2MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 67.3MB / 2.49GB  ...nts/optim_090000_rank2.pt: 1%| | 20.8MB / 2.49GB  ...nts/optim_090000_rank4.pt: 1%| | 13.0MB / 2.49GB  ...nts/optim_090000_rank5.pt: 1%| | 21.7MB / 2.49GB  ...nts/optim_090000_rank6.pt: 2%|▏ | 50.2MB / 2.49GB  ...nts/optim_090000_rank0.pt: 2%|▏ | 48.4MB / 2.49GB  ...nts/optim_090000_rank7.pt: 2%|▏ | 46.7MB / 2.49GB  ...eckpoints/model_090000.pt: 1%| | 23.9MB / 2.35GB  Processing Files (0 / 8) : 1%|▏ | 292MB / 22.2GB, 36.5MB/s New Data Upload : 23%|██▎ | 234MB / 1.01GB, 29.3MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 67.4MB / 2.49GB  ...nts/optim_090000_rank2.pt: 1%| | 21.8MB / 2.49GB  ...nts/optim_090000_rank4.pt: 1%| | 13.1MB / 2.49GB  ...nts/optim_090000_rank5.pt: 1%| | 23.6MB / 2.49GB  ...nts/optim_090000_rank6.pt: 2%|▏ | 50.4MB / 2.49GB  ...nts/optim_090000_rank0.pt: 2%|▏ | 53.3MB / 2.49GB  ...nts/optim_090000_rank7.pt: 2%|▏ | 49.2MB / 2.49GB  ...eckpoints/model_090000.pt: 1%|▏ | 29.7MB / 2.35GB  Processing Files (0 / 8) : 1%|▏ | 309MB / 22.2GB, 37.6MB/s New Data Upload : 25%|██▍ | 249MB / 1.01GB, 30.4MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 67.6MB / 2.49GB  ...nts/optim_090000_rank2.pt: 1%| | 24.3MB / 2.49GB  ...nts/optim_090000_rank4.pt: 1%| | 15.8MB / 2.49GB  ...nts/optim_090000_rank5.pt: 1%| | 28.8MB / 2.49GB  ...nts/optim_090000_rank6.pt: 2%|▏ | 54.9MB / 2.49GB  ...nts/optim_090000_rank0.pt: 2%|▏ | 59.0MB / 2.49GB  ...nts/optim_090000_rank7.pt: 2%|▏ | 57.3MB / 2.49GB  ...eckpoints/model_090000.pt: 1%|▏ | 34.4MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 342MB / 22.2GB, 40.7MB/s New Data Upload : 28%|██▊ | 277MB / 1.01GB, 33.0MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 67.8MB / 2.49GB  ...nts/optim_090000_rank2.pt: 1%| | 25.1MB / 2.49GB  ...nts/optim_090000_rank4.pt: 1%| | 16.6MB / 2.49GB  ...nts/optim_090000_rank5.pt: 1%|▏ | 31.3MB / 2.49GB  ...nts/optim_090000_rank6.pt: 2%|▏ | 57.3MB / 2.49GB  ...nts/optim_090000_rank0.pt: 3%|▎ | 64.9MB / 2.49GB  ...nts/optim_090000_rank7.pt: 2%|▏ | 61.4MB / 2.49GB  ...eckpoints/model_090000.pt: 2%|▏ | 37.3MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 362MB / 22.2GB, 42.0MB/s New Data Upload : 29%|██▉ | 295MB / 1.01GB, 34.3MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 71.8MB / 2.49GB  ...nts/optim_090000_rank2.pt: 1%| | 27.7MB / 2.49GB  ...nts/optim_090000_rank4.pt: 1%| | 20.3MB / 2.49GB  ...nts/optim_090000_rank5.pt: 2%|▏ | 41.0MB / 2.49GB  ...nts/optim_090000_rank6.pt: 2%|▏ | 60.5MB / 2.49GB  ...nts/optim_090000_rank0.pt: 3%|▎ | 67.6MB / 2.49GB  ...nts/optim_090000_rank7.pt: 3%|▎ | 67.6MB / 2.49GB  ...eckpoints/model_090000.pt: 2%|▏ | 44.8MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 401MB / 22.2GB, 45.6MB/s New Data Upload : 30%|███ | 322MB / 1.07GB, 36.6MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 72.2MB / 2.49GB  ...nts/optim_090000_rank2.pt: 1%| | 28.3MB / 2.49GB  ...nts/optim_090000_rank4.pt: 1%| | 22.2MB / 2.49GB  ...nts/optim_090000_rank5.pt: 2%|▏ | 47.6MB / 2.49GB  ...nts/optim_090000_rank6.pt: 3%|▎ | 62.5MB / 2.49GB  ...nts/optim_090000_rank0.pt: 3%|▎ | 69.5MB / 2.49GB  ...nts/optim_090000_rank7.pt: 3%|▎ | 70.1MB / 2.49GB  ...eckpoints/model_090000.pt: 2%|▏ | 48.3MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 421MB / 22.2GB, 46.7MB/s New Data Upload : 32%|███▏ | 340MB / 1.07GB, 37.8MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 72.8MB / 2.49GB  ...nts/optim_090000_rank2.pt: 1%| | 29.8MB / 2.49GB  ...nts/optim_090000_rank4.pt: 1%| | 25.9MB / 2.49GB  ...nts/optim_090000_rank5.pt: 2%|▏ | 52.6MB / 2.49GB  ...nts/optim_090000_rank6.pt: 3%|▎ | 64.5MB / 2.49GB  ...nts/optim_090000_rank0.pt: 3%|▎ | 69.8MB / 2.49GB  ...nts/optim_090000_rank7.pt: 3%|▎ | 75.1MB / 2.49GB  ...eckpoints/model_090000.pt: 2%|▏ | 55.2MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 446MB / 22.2GB, 48.5MB/s New Data Upload : 34%|███▍ | 362MB / 1.07GB, 39.4MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 76.5MB / 2.49GB  ...nts/optim_090000_rank2.pt: 1%|▏ | 33.2MB / 2.49GB  ...nts/optim_090000_rank4.pt: 1%|▏ | 32.2MB / 2.49GB  ...nts/optim_090000_rank5.pt: 2%|▏ | 55.6MB / 2.49GB  ...nts/optim_090000_rank6.pt: 3%|▎ | 69.4MB / 2.49GB  ...nts/optim_090000_rank0.pt: 3%|▎ | 75.1MB / 2.49GB  ...nts/optim_090000_rank7.pt: 3%|▎ | 78.2MB / 2.49GB  ...eckpoints/model_090000.pt: 3%|▎ | 59.8MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 480MB / 22.2GB, 51.1MB/s New Data Upload : 35%|███▌ | 376MB / 1.07GB, 40.0MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 81.8MB / 2.49GB  ...nts/optim_090000_rank2.pt: 2%|▏ | 37.7MB / 2.49GB  ...nts/optim_090000_rank4.pt: 2%|▏ | 41.3MB / 2.49GB  ...nts/optim_090000_rank5.pt: 3%|▎ | 63.8MB / 2.49GB  ...nts/optim_090000_rank6.pt: 3%|▎ | 75.5MB / 2.49GB  ...nts/optim_090000_rank0.pt: 3%|▎ | 79.5MB / 2.49GB  ...nts/optim_090000_rank7.pt: 3%|▎ | 82.6MB / 2.49GB  ...eckpoints/model_090000.pt: 3%|▎ | 62.2MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 524MB / 22.2GB, 54.6MB/s New Data Upload : 35%|███▍ | 394MB / 1.14GB, 41.1MB/s  ...nts/optim_090000_rank1.pt: 3%|▎ | 84.8MB / 2.49GB  ...nts/optim_090000_rank2.pt: 2%|▏ | 39.9MB / 2.49GB  ...nts/optim_090000_rank4.pt: 2%|▏ | 46.9MB / 2.49GB  ...nts/optim_090000_rank5.pt: 3%|▎ | 66.2MB / 2.49GB  ...nts/optim_090000_rank6.pt: 3%|▎ | 81.9MB / 2.49GB  ...nts/optim_090000_rank0.pt: 3%|▎ | 81.9MB / 2.49GB  ...nts/optim_090000_rank7.pt: 3%|▎ | 85.0MB / 2.49GB  ...eckpoints/model_090000.pt: 3%|▎ | 67.4MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 554MB / 22.2GB, 56.5MB/s New Data Upload : 36%|███▌ | 407MB / 1.14GB, 41.6MB/s  ...nts/optim_090000_rank1.pt: 4%|▍ | 96.9MB / 2.49GB  ...nts/optim_090000_rank2.pt: 2%|▏ | 51.8MB / 2.49GB  ...nts/optim_090000_rank4.pt: 3%|▎ | 62.2MB / 2.49GB  ...nts/optim_090000_rank5.pt: 3%|▎ | 82.4MB / 2.49GB  ...nts/optim_090000_rank6.pt: 4%|▍ | 93.2MB / 2.49GB  ...nts/optim_090000_rank0.pt: 4%|▎ | 92.5MB / 2.49GB  ...nts/optim_090000_rank7.pt: 4%|▍ | 96.8MB / 2.49GB  ...eckpoints/model_090000.pt: 3%|▎ | 69.7MB / 2.35GB  Processing Files (0 / 8) : 3%|▎ | 645MB / 22.2GB, 64.5MB/s New Data Upload : 38%|███▊ | 432MB / 1.14GB, 43.2MB/s  ...nts/optim_090000_rank1.pt: 4%|▍ | 107MB / 2.49GB  ...nts/optim_090000_rank2.pt: 2%|▏ | 60.8MB / 2.49GB  ...nts/optim_090000_rank4.pt: 3%|▎ | 74.0MB / 2.49GB  ...nts/optim_090000_rank5.pt: 4%|▍ | 94.8MB / 2.49GB  ...nts/optim_090000_rank6.pt: 4%|▍ | 102MB / 2.49GB  ...nts/optim_090000_rank0.pt: 4%|▍ | 103MB / 2.49GB  ...nts/optim_090000_rank7.pt: 4%|▍ | 106MB / 2.49GB  ...eckpoints/model_090000.pt: 3%|▎ | 73.6MB / 2.35GB  Processing Files (0 / 8) : 3%|▎ | 721MB / 22.2GB, 70.6MB/s New Data Upload : 36%|███▌ | 457MB / 1.27GB, 44.8MB/s  ...nts/optim_090000_rank1.pt: 5%|▍ | 123MB / 2.49GB  ...nts/optim_090000_rank2.pt: 3%|▎ | 75.1MB / 2.49GB  ...nts/optim_090000_rank4.pt: 4%|▍ | 95.2MB / 2.49GB  ...nts/optim_090000_rank5.pt: 4%|▍ | 112MB / 2.49GB  ...nts/optim_090000_rank6.pt: 5%|▍ | 116MB / 2.49GB  ...nts/optim_090000_rank0.pt: 5%|▍ | 121MB / 2.49GB  ...nts/optim_090000_rank7.pt: 5%|▍ | 120MB / 2.49GB  ...eckpoints/model_090000.pt: 3%|▎ | 73.6MB / 2.35GB  Processing Files (0 / 8) : 4%|▍ | 836MB / 22.2GB, 81.9MB/s New Data Upload : 38%|███▊ | 484MB / 1.27GB, 47.4MB/s  ...nts/optim_090000_rank1.pt: 6%|▌ | 140MB / 2.49GB  ...nts/optim_090000_rank2.pt: 4%|▎ | 90.9MB / 2.49GB  ...nts/optim_090000_rank4.pt: 5%|▍ | 115MB / 2.49GB  ...nts/optim_090000_rank5.pt: 5%|▌ | 129MB / 2.49GB  ...nts/optim_090000_rank6.pt: 5%|▌ | 132MB / 2.49GB  ...nts/optim_090000_rank0.pt: 6%|▌ | 139MB / 2.49GB  ...nts/optim_090000_rank7.pt: 5%|▌ | 136MB / 2.49GB  ...eckpoints/model_090000.pt: 3%|▎ | 75.8MB / 2.35GB  Processing Files (0 / 8) : 4%|▍ | 957MB / 22.2GB, 93.9MB/s New Data Upload : 38%|███▊ | 513MB / 1.34GB, 50.3MB/s  ...nts/optim_090000_rank1.pt: 6%|▋ | 157MB / 2.49GB  ...nts/optim_090000_rank2.pt: 4%|▍ | 108MB / 2.49GB  ...nts/optim_090000_rank4.pt: 5%|▌ | 136MB / 2.49GB  ...nts/optim_090000_rank5.pt: 6%|▌ | 148MB / 2.49GB  ...nts/optim_090000_rank6.pt: 6%|▌ | 149MB / 2.49GB  ...nts/optim_090000_rank0.pt: 6%|▋ | 156MB / 2.49GB  ...nts/optim_090000_rank7.pt: 6%|▌ | 154MB / 2.49GB  ...eckpoints/model_090000.pt: 3%|▎ | 80.2MB / 2.35GB  Processing Files (0 / 8) : 5%|▍ | 1.09GB / 22.2GB, 107MB/s New Data Upload : 39%|███▉ | 546MB / 1.41GB, 53.6MB/s  ...nts/optim_090000_rank1.pt: 7%|▋ | 177MB / 2.49GB  ...nts/optim_090000_rank2.pt: 5%|▌ | 127MB / 2.49GB  ...nts/optim_090000_rank4.pt: 6%|▋ | 158MB / 2.49GB  ...nts/optim_090000_rank5.pt: 7%|▋ | 171MB / 2.49GB  ...nts/optim_090000_rank6.pt: 7%|▋ | 168MB / 2.49GB  ...nts/optim_090000_rank0.pt: 7%|▋ | 174MB / 2.49GB  ...nts/optim_090000_rank7.pt: 7%|▋ | 173MB / 2.49GB  ...eckpoints/model_090000.pt: 4%|▎ | 83.0MB / 2.35GB  Processing Files (0 / 8) : 6%|▌ | 1.23GB / 22.2GB, 121MB/s New Data Upload : 39%|███▉ | 580MB / 1.48GB, 56.8MB/s  ...nts/optim_090000_rank1.pt: 8%|▊ | 197MB / 2.49GB  ...nts/optim_090000_rank2.pt: 6%|▌ | 146MB / 2.49GB  ...nts/optim_090000_rank4.pt: 7%|▋ | 181MB / 2.49GB  ...nts/optim_090000_rank5.pt: 8%|▊ | 190MB / 2.49GB  ...nts/optim_090000_rank6.pt: 8%|▊ | 187MB / 2.49GB  ...nts/optim_090000_rank0.pt: 8%|▊ | 197MB / 2.49GB  ...nts/optim_090000_rank7.pt: 8%|▊ | 192MB / 2.49GB  ...eckpoints/model_090000.pt: 4%|▎ | 86.3MB / 2.35GB  Processing Files (0 / 8) : 6%|▌ | 1.38GB / 22.2GB, 135MB/s New Data Upload : 42%|████▏ | 615MB / 1.48GB, 60.2MB/s  ...nts/optim_090000_rank1.pt: 9%|▉ | 225MB / 2.49GB  ...nts/optim_090000_rank2.pt: 7%|▋ | 175MB / 2.49GB  ...nts/optim_090000_rank4.pt: 9%|▊ | 213MB / 2.49GB  ...nts/optim_090000_rank5.pt: 9%|▉ | 220MB / 2.49GB  ...nts/optim_090000_rank6.pt: 9%|▊ | 215MB / 2.49GB  ...nts/optim_090000_rank0.pt: 9%|▉ | 221MB / 2.49GB  ...nts/optim_090000_rank7.pt: 9%|▉ | 220MB / 2.49GB  ...eckpoints/model_090000.pt: 4%|▍ | 90.2MB / 2.35GB  Processing Files (0 / 8) : 7%|▋ | 1.58GB / 22.2GB, 154MB/s New Data Upload : 41%|████ | 655MB / 1.61GB, 64.1MB/s  ...nts/optim_090000_rank1.pt: 10%|▉ | 245MB / 2.49GB  ...nts/optim_090000_rank2.pt: 8%|▊ | 195MB / 2.49GB  ...nts/optim_090000_rank4.pt: 9%|▉ | 235MB / 2.49GB  ...nts/optim_090000_rank5.pt: 10%|▉ | 241MB / 2.49GB  ...nts/optim_090000_rank6.pt: 9%|▉ | 234MB / 2.49GB  ...nts/optim_090000_rank0.pt: 10%|▉ | 241MB / 2.49GB  ...nts/optim_090000_rank7.pt: 10%|▉ | 240MB / 2.49GB  ...eckpoints/model_090000.pt: 4%|▍ | 90.2MB / 2.35GB  Processing Files (0 / 8) : 8%|▊ | 1.72GB / 22.2GB, 168MB/s New Data Upload : 41%|████ | 685MB / 1.68GB, 66.5MB/s  ...nts/optim_090000_rank1.pt: 11%|█ | 262MB / 2.49GB  ...nts/optim_090000_rank2.pt: 9%|▊ | 212MB / 2.49GB  ...nts/optim_090000_rank4.pt: 10%|█ | 255MB / 2.49GB  ...nts/optim_090000_rank5.pt: 10%|█ | 259MB / 2.49GB  ...nts/optim_090000_rank6.pt: 10%|█ | 251MB / 2.49GB  ...nts/optim_090000_rank0.pt: 10%|█ | 256MB / 2.49GB  ...nts/optim_090000_rank7.pt: 10%|█ | 256MB / 2.49GB  ...eckpoints/model_090000.pt: 4%|▍ | 94.7MB / 2.35GB  Processing Files (0 / 8) : 8%|▊ | 1.85GB / 22.2GB, 180MB/s New Data Upload : 43%|████▎ | 716MB / 1.68GB, 69.2MB/s  ...nts/optim_090000_rank1.pt: 12%|█▏ | 288MB / 2.49GB  ...nts/optim_090000_rank2.pt: 10%|▉ | 239MB / 2.49GB  ...nts/optim_090000_rank4.pt: 11%|█▏ | 285MB / 2.49GB  ...nts/optim_090000_rank5.pt: 12%|█▏ | 286MB / 2.49GB  ...nts/optim_090000_rank6.pt: 11%|█ | 277MB / 2.49GB  ...nts/optim_090000_rank0.pt: 11%|█▏ | 284MB / 2.49GB  ...nts/optim_090000_rank7.pt: 11%|█▏ | 282MB / 2.49GB  ...eckpoints/model_090000.pt: 4%|▍ | 101MB / 2.35GB  Processing Files (0 / 8) : 9%|▉ | 2.04GB / 22.2GB, 198MB/s New Data Upload : 42%|████▏ | 762MB / 1.81GB, 73.2MB/s  ...nts/optim_090000_rank1.pt: 12%|█▏ | 309MB / 2.49GB  ...nts/optim_090000_rank2.pt: 10%|█ | 260MB / 2.49GB  ...nts/optim_090000_rank4.pt: 12%|█▏ | 306MB / 2.49GB  ...nts/optim_090000_rank5.pt: 12%|█▏ | 307MB / 2.49GB  ...nts/optim_090000_rank6.pt: 12%|█▏ | 297MB / 2.49GB  ...nts/optim_090000_rank0.pt: 12%|█▏ | 303MB / 2.49GB  ...nts/optim_090000_rank7.pt: 12%|█▏ | 302MB / 2.49GB  ...eckpoints/model_090000.pt: 4%|▍ | 103MB / 2.35GB  Processing Files (0 / 8) : 10%|▉ | 2.19GB / 22.2GB, 212MB/s New Data Upload : 43%|████▎ | 786MB / 1.81GB, 75.3MB/s  ...nts/optim_090000_rank1.pt: 13%|█▎ | 335MB / 2.49GB  ...nts/optim_090000_rank2.pt: 12%|█▏ | 287MB / 2.49GB  ...nts/optim_090000_rank4.pt: 13%|█▎ | 332MB / 2.49GB  ...nts/optim_090000_rank5.pt: 13%|█▎ | 334MB / 2.49GB  ...nts/optim_090000_rank6.pt: 13%|█▎ | 323MB / 2.49GB  ...nts/optim_090000_rank0.pt: 13%|█▎ | 327MB / 2.49GB  ...nts/optim_090000_rank7.pt: 13%|█▎ | 329MB / 2.49GB  ...eckpoints/model_090000.pt: 5%|▍ | 106MB / 2.35GB  Processing Files (0 / 8) : 11%|█ | 2.37GB / 22.2GB, 229MB/s New Data Upload : 46%|████▌ | 824MB / 1.81GB, 78.5MB/s  ...nts/optim_090000_rank1.pt: 14%|█▍ | 355MB / 2.49GB  ...nts/optim_090000_rank2.pt: 12%|█▏ | 307MB / 2.49GB  ...nts/optim_090000_rank4.pt: 14%|█▍ | 352MB / 2.49GB  ...nts/optim_090000_rank5.pt: 14%|█▍ | 355MB / 2.49GB  ...nts/optim_090000_rank6.pt: 14%|█▍ | 343MB / 2.49GB  ...nts/optim_090000_rank0.pt: 14%|█▍ | 346MB / 2.49GB  ...nts/optim_090000_rank7.pt: 14%|█▍ | 349MB / 2.49GB  ...eckpoints/model_090000.pt: 5%|▍ | 110MB / 2.35GB  Processing Files (0 / 8) : 11%|█▏ | 2.52GB / 22.2GB, 243MB/s New Data Upload : 47%|████▋ | 853MB / 1.81GB, 81.0MB/s  ...nts/optim_090000_rank1.pt: 15%|█▌ | 380MB / 2.49GB  ...nts/optim_090000_rank2.pt: 13%|█▎ | 334MB / 2.49GB  ...nts/optim_090000_rank4.pt: 15%|█▌ | 378MB / 2.49GB  ...nts/optim_090000_rank5.pt: 15%|█▌ | 381MB / 2.49GB  ...nts/optim_090000_rank6.pt: 15%|█▍ | 369MB / 2.49GB  ...nts/optim_090000_rank0.pt: 15%|█▍ | 367MB / 2.49GB  ...nts/optim_090000_rank7.pt: 15%|█▌ | 374MB / 2.49GB  ...eckpoints/model_090000.pt: 5%|▍ | 114MB / 2.35GB  Processing Files (0 / 8) : 12%|█▏ | 2.70GB / 22.2GB, 260MB/s New Data Upload : 49%|████▉ | 885MB / 1.81GB, 83.6MB/s  ...nts/optim_090000_rank1.pt: 16%|█▋ | 405MB / 2.49GB  ...nts/optim_090000_rank2.pt: 14%|█▍ | 359MB / 2.49GB  ...nts/optim_090000_rank4.pt: 16%|█▌ | 403MB / 2.49GB  ...nts/optim_090000_rank5.pt: 16%|█▋ | 406MB / 2.49GB  ...nts/optim_090000_rank6.pt: 16%|█▌ | 394MB / 2.49GB  ...nts/optim_090000_rank0.pt: 16%|█▌ | 390MB / 2.49GB  ...nts/optim_090000_rank7.pt: 16%|█▌ | 401MB / 2.49GB  ...eckpoints/model_090000.pt: 5%|▍ | 114MB / 2.35GB  Processing Files (0 / 8) : 13%|█▎ | 2.87GB / 22.2GB, 276MB/s New Data Upload : 51%|█████ | 915MB / 1.81GB, 86.2MB/s  ...nts/optim_090000_rank1.pt: 17%|█▋ | 431MB / 2.49GB  ...nts/optim_090000_rank2.pt: 15%|█▌ | 384MB / 2.49GB  ...nts/optim_090000_rank4.pt: 17%|█▋ | 427MB / 2.49GB  ...nts/optim_090000_rank5.pt: 17%|█▋ | 430MB / 2.49GB  ...nts/optim_090000_rank6.pt: 17%|█▋ | 418MB / 2.49GB  ...nts/optim_090000_rank0.pt: 17%|█▋ | 414MB / 2.49GB  ...nts/optim_090000_rank7.pt: 17%|█▋ | 425MB / 2.49GB  ...eckpoints/model_090000.pt: 5%|▍ | 117MB / 2.35GB  Processing Files (0 / 8) : 14%|█▎ | 3.05GB / 22.2GB, 293MB/s New Data Upload : 51%|█████ | 949MB / 1.88GB, 89.3MB/s  ...nts/optim_090000_rank1.pt: 18%|█▊ | 453MB / 2.49GB  ...nts/optim_090000_rank2.pt: 16%|█▋ | 407MB / 2.49GB  ...nts/optim_090000_rank4.pt: 18%|█▊ | 450MB / 2.49GB  ...nts/optim_090000_rank5.pt: 18%|█▊ | 452MB / 2.49GB  ...nts/optim_090000_rank6.pt: 18%|█▊ | 441MB / 2.49GB  ...nts/optim_090000_rank0.pt: 18%|█▊ | 436MB / 2.49GB  ...nts/optim_090000_rank7.pt: 18%|█▊ | 448MB / 2.49GB  ...eckpoints/model_090000.pt: 5%|▌ | 122MB / 2.35GB  Processing Files (0 / 8) : 14%|█▍ | 3.21GB / 22.2GB, 308MB/s New Data Upload : 52%|█████▏ | 982MB / 1.88GB, 92.1MB/s  ...nts/optim_090000_rank1.pt: 19%|█▉ | 472MB / 2.49GB  ...nts/optim_090000_rank2.pt: 17%|█▋ | 426MB / 2.49GB  ...nts/optim_090000_rank4.pt: 19%|█▉ | 469MB / 2.49GB  ...nts/optim_090000_rank5.pt: 19%|█▉ | 470MB / 2.49GB  ...nts/optim_090000_rank6.pt: 18%|█▊ | 459MB / 2.49GB  ...nts/optim_090000_rank0.pt: 18%|█▊ | 458MB / 2.49GB  ...nts/optim_090000_rank7.pt: 19%|█▊ | 465MB / 2.49GB  ...eckpoints/model_090000.pt: 5%|▌ | 124MB / 2.35GB  Processing Files (0 / 8) : 15%|█▌ | 3.34GB / 22.2GB, 321MB/s New Data Upload : 54%|█████▎ | 1.01GB / 1.88GB, 94.3MB/s  ...nts/optim_090000_rank1.pt: 20%|█▉ | 495MB / 2.49GB  ...nts/optim_090000_rank2.pt: 18%|█▊ | 451MB / 2.49GB  ...nts/optim_090000_rank4.pt: 20%|█▉ | 491MB / 2.49GB  ...nts/optim_090000_rank5.pt: 20%|█▉ | 494MB / 2.49GB  ...nts/optim_090000_rank6.pt: 19%|█▉ | 483MB / 2.49GB  ...nts/optim_090000_rank0.pt: 19%|█▉ | 480MB / 2.49GB  ...nts/optim_090000_rank7.pt: 20%|█▉ | 488MB / 2.49GB  ...eckpoints/model_090000.pt: 5%|▌ | 126MB / 2.35GB  Processing Files (0 / 8) : 16%|█▌ | 3.51GB / 22.2GB, 337MB/s New Data Upload : 53%|█████▎ | 1.04GB / 1.94GB, 97.0MB/s  ...nts/optim_090000_rank1.pt: 21%|██ | 519MB / 2.49GB  ...nts/optim_090000_rank2.pt: 19%|█▉ | 474MB / 2.49GB  ...nts/optim_090000_rank4.pt: 21%|██ | 513MB / 2.49GB  ...nts/optim_090000_rank5.pt: 21%|██ | 516MB / 2.49GB  ...nts/optim_090000_rank6.pt: 20%|██ | 505MB / 2.49GB  ...nts/optim_090000_rank0.pt: 20%|██ | 503MB / 2.49GB  ...nts/optim_090000_rank7.pt: 20%|██ | 506MB / 2.49GB  ...eckpoints/model_090000.pt: 6%|▌ | 129MB / 2.35GB  Processing Files (0 / 8) : 16%|█▋ | 3.66GB / 22.2GB, 351MB/s New Data Upload : 53%|█████▎ | 1.07GB / 2.01GB, 99.6MB/s  ...nts/optim_090000_rank1.pt: 22%|██▏ | 537MB / 2.49GB  ...nts/optim_090000_rank2.pt: 20%|█▉ | 492MB / 2.49GB  ...nts/optim_090000_rank4.pt: 21%|██▏ | 530MB / 2.49GB  ...nts/optim_090000_rank5.pt: 21%|██▏ | 533MB / 2.49GB  ...nts/optim_090000_rank6.pt: 21%|██ | 522MB / 2.49GB  ...nts/optim_090000_rank0.pt: 21%|██ | 522MB / 2.49GB  ...nts/optim_090000_rank7.pt: 21%|██ | 521MB / 2.49GB  ...eckpoints/model_090000.pt: 6%|▌ | 131MB / 2.35GB  Processing Files (0 / 8) : 17%|█▋ | 3.79GB / 22.2GB, 363MB/s New Data Upload : 48%|████▊ | 1.09GB / 2.28GB, 102MB/s  ...nts/optim_090000_rank1.pt: 23%|██▎ | 565MB / 2.49GB  ...nts/optim_090000_rank2.pt: 21%|██ | 521MB / 2.49GB  ...nts/optim_090000_rank4.pt: 22%|██▏ | 558MB / 2.49GB  ...nts/optim_090000_rank5.pt: 23%|██▎ | 561MB / 2.49GB  ...nts/optim_090000_rank6.pt: 22%|██▏ | 550MB / 2.49GB  ...nts/optim_090000_rank0.pt: 22%|██▏ | 552MB / 2.49GB  ...nts/optim_090000_rank7.pt: 22%|██▏ | 546MB / 2.49GB  ...eckpoints/model_090000.pt: 6%|▌ | 136MB / 2.35GB  Processing Files (0 / 8) : 18%|█▊ | 3.99GB / 22.2GB, 382MB/s New Data Upload : 48%|████▊ | 1.14GB / 2.35GB, 105MB/s  ...nts/optim_090000_rank1.pt: 24%|██▎ | 590MB / 2.49GB  ...nts/optim_090000_rank2.pt: 22%|██▏ | 546MB / 2.49GB  ...nts/optim_090000_rank4.pt: 24%|██▎ | 586MB / 2.49GB  ...nts/optim_090000_rank5.pt: 24%|██▎ | 587MB / 2.49GB  ...nts/optim_090000_rank6.pt: 23%|██▎ | 576MB / 2.49GB  ...nts/optim_090000_rank0.pt: 23%|██▎ | 578MB / 2.49GB  ...nts/optim_090000_rank7.pt: 23%|██▎ | 568MB / 2.49GB  ...eckpoints/model_090000.pt: 6%|▌ | 142MB / 2.35GB  Processing Files (0 / 8) : 19%|█▉ | 4.17GB / 22.2GB, 400MB/s New Data Upload : 49%|████▊ | 1.17GB / 2.41GB, 109MB/s  ...nts/optim_090000_rank1.pt: 24%|██▍ | 608MB / 2.49GB  ...nts/optim_090000_rank2.pt: 23%|██▎ | 564MB / 2.49GB  ...nts/optim_090000_rank4.pt: 24%|██▍ | 606MB / 2.49GB  ...nts/optim_090000_rank5.pt: 24%|██▍ | 606MB / 2.49GB  ...nts/optim_090000_rank6.pt: 24%|██▍ | 595MB / 2.49GB  ...nts/optim_090000_rank0.pt: 24%|██▍ | 599MB / 2.49GB  ...nts/optim_090000_rank7.pt: 24%|██▎ | 586MB / 2.49GB  ...eckpoints/model_090000.pt: 6%|▋ | 148MB / 2.35GB  Processing Files (0 / 8) : 19%|█▉ | 4.31GB / 22.2GB, 413MB/s New Data Upload : 48%|████▊ | 1.20GB / 2.48GB, 111MB/s  ...nts/optim_090000_rank1.pt: 25%|██▌ | 629MB / 2.49GB  ...nts/optim_090000_rank2.pt: 24%|██▎ | 585MB / 2.49GB  ...nts/optim_090000_rank4.pt: 25%|██▌ | 629MB / 2.49GB  ...nts/optim_090000_rank5.pt: 25%|██▌ | 628MB / 2.49GB  ...nts/optim_090000_rank6.pt: 25%|██▍ | 617MB / 2.49GB  ...nts/optim_090000_rank0.pt: 25%|██▌ | 622MB / 2.49GB  ...nts/optim_090000_rank7.pt: 24%|██▍ | 606MB / 2.49GB  ...eckpoints/model_090000.pt: 6%|▋ | 152MB / 2.35GB  Processing Files (0 / 8) : 20%|██ | 4.47GB / 22.2GB, 428MB/s New Data Upload : 50%|████▉ | 1.23GB / 2.48GB, 114MB/s  ...nts/optim_090000_rank1.pt: 26%|██▌ | 652MB / 2.49GB  ...nts/optim_090000_rank2.pt: 25%|██▍ | 610MB / 2.49GB  ...nts/optim_090000_rank4.pt: 26%|██▋ | 654MB / 2.49GB  ...nts/optim_090000_rank5.pt: 26%|██▋ | 653MB / 2.49GB  ...nts/optim_090000_rank6.pt: 26%|██▌ | 641MB / 2.49GB  ...nts/optim_090000_rank0.pt: 26%|██▌ | 647MB / 2.49GB  ...nts/optim_090000_rank7.pt: 25%|██▌ | 628MB / 2.49GB  ...eckpoints/model_090000.pt: 7%|▋ | 158MB / 2.35GB  Processing Files (0 / 8) : 21%|██ | 4.64GB / 22.2GB, 445MB/s New Data Upload : 51%|█████ | 1.27GB / 2.48GB, 118MB/s  ...nts/optim_090000_rank1.pt: 27%|██▋ | 670MB / 2.49GB  ...nts/optim_090000_rank2.pt: 25%|██▌ | 629MB / 2.49GB  ...nts/optim_090000_rank4.pt: 27%|██▋ | 676MB / 2.49GB  ...nts/optim_090000_rank5.pt: 27%|██▋ | 674MB / 2.49GB  ...nts/optim_090000_rank6.pt: 27%|██▋ | 660MB / 2.49GB  ...nts/optim_090000_rank0.pt: 27%|██▋ | 671MB / 2.49GB  ...nts/optim_090000_rank7.pt: 26%|██▌ | 647MB / 2.49GB  ...eckpoints/model_090000.pt: 7%|▋ | 163MB / 2.35GB  Processing Files (0 / 8) : 22%|██▏ | 4.79GB / 22.2GB, 460MB/s New Data Upload : 51%|█████ | 1.30GB / 2.55GB, 121MB/s  ...nts/optim_090000_rank1.pt: 28%|██▊ | 691MB / 2.49GB  ...nts/optim_090000_rank2.pt: 26%|██▌ | 651MB / 2.49GB  ...nts/optim_090000_rank4.pt: 28%|██▊ | 701MB / 2.49GB  ...nts/optim_090000_rank5.pt: 28%|██▊ | 697MB / 2.49GB  ...nts/optim_090000_rank6.pt: 27%|██▋ | 682MB / 2.49GB  ...nts/optim_090000_rank0.pt: 28%|██▊ | 696MB / 2.49GB  ...nts/optim_090000_rank7.pt: 27%|██▋ | 669MB / 2.49GB  ...eckpoints/model_090000.pt: 7%|▋ | 166MB / 2.35GB  Processing Files (0 / 8) : 22%|██▏ | 4.95GB / 22.2GB, 475MB/s New Data Upload : 53%|█████▎ | 1.34GB / 2.55GB, 124MB/s  ...nts/optim_090000_rank1.pt: 29%|██▊ | 713MB / 2.49GB  ...nts/optim_090000_rank2.pt: 27%|██▋ | 669MB / 2.49GB  ...nts/optim_090000_rank4.pt: 29%|██▉ | 722MB / 2.49GB  ...nts/optim_090000_rank5.pt: 29%|██▉ | 717MB / 2.49GB  ...nts/optim_090000_rank6.pt: 28%|██▊ | 702MB / 2.49GB  ...nts/optim_090000_rank0.pt: 29%|██▉ | 718MB / 2.49GB  ...nts/optim_090000_rank7.pt: 28%|██▊ | 690MB / 2.49GB  ...eckpoints/model_090000.pt: 7%|▋ | 173MB / 2.35GB  Processing Files (0 / 8) : 23%|██▎ | 5.10GB / 22.2GB, 490MB/s New Data Upload : 54%|█████▍ | 1.38GB / 2.55GB, 128MB/s  ...nts/optim_090000_rank1.pt: 29%|██▉ | 728MB / 2.49GB  ...nts/optim_090000_rank2.pt: 28%|██▊ | 686MB / 2.49GB  ...nts/optim_090000_rank4.pt: 30%|██▉ | 740MB / 2.49GB  ...nts/optim_090000_rank5.pt: 30%|██▉ | 735MB / 2.49GB  ...nts/optim_090000_rank6.pt: 29%|██▉ | 719MB / 2.49GB  ...nts/optim_090000_rank0.pt: 30%|██▉ | 737MB / 2.49GB  ...nts/optim_090000_rank7.pt: 29%|██▊ | 710MB / 2.49GB  ...eckpoints/model_090000.pt: 7%|▋ | 174MB / 2.35GB  Processing Files (0 / 8) : 24%|██▎ | 5.23GB / 22.2GB, 502MB/s New Data Upload : 54%|█████▍ | 1.41GB / 2.62GB, 130MB/s  ...nts/optim_090000_rank1.pt: 30%|██▉ | 737MB / 2.49GB  ...nts/optim_090000_rank2.pt: 28%|██▊ | 696MB / 2.49GB  ...nts/optim_090000_rank4.pt: 30%|███ | 749MB / 2.49GB  ...nts/optim_090000_rank5.pt: 30%|██▉ | 744MB / 2.49GB  ...nts/optim_090000_rank6.pt: 29%|██▉ | 727MB / 2.49GB  ...nts/optim_090000_rank0.pt: 30%|███ | 747MB / 2.49GB  ...nts/optim_090000_rank7.pt: 29%|██▉ | 717MB / 2.49GB  ...eckpoints/model_090000.pt: 8%|▊ | 180MB / 2.35GB  Processing Files (0 / 8) : 24%|██▍ | 5.30GB / 22.2GB, 508MB/s New Data Upload : 54%|█████▍ | 1.42GB / 2.62GB, 132MB/s  ...nts/optim_090000_rank1.pt: 31%|███ | 760MB / 2.49GB  ...nts/optim_090000_rank2.pt: 29%|██▉ | 719MB / 2.49GB  ...nts/optim_090000_rank4.pt: 31%|███ | 775MB / 2.49GB  ...nts/optim_090000_rank5.pt: 31%|███ | 767MB / 2.49GB  ...nts/optim_090000_rank6.pt: 30%|███ | 751MB / 2.49GB  ...nts/optim_090000_rank0.pt: 31%|███ | 775MB / 2.49GB  ...nts/optim_090000_rank7.pt: 30%|██▉ | 743MB / 2.49GB  ...eckpoints/model_090000.pt: 8%|▊ | 184MB / 2.35GB  Processing Files (0 / 8) : 25%|██▍ | 5.47GB / 22.2GB, 524MB/s New Data Upload : 55%|█████▍ | 1.47GB / 2.68GB, 135MB/s  ...nts/optim_090000_rank1.pt: 31%|███▏ | 779MB / 2.49GB  ...nts/optim_090000_rank2.pt: 30%|██▉ | 738MB / 2.49GB  ...nts/optim_090000_rank4.pt: 32%|███▏ | 793MB / 2.49GB  ...nts/optim_090000_rank5.pt: 32%|███▏ | 786MB / 2.49GB  ...nts/optim_090000_rank6.pt: 31%|███ | 768MB / 2.49GB  ...nts/optim_090000_rank0.pt: 32%|███▏ | 796MB / 2.49GB  ...nts/optim_090000_rank7.pt: 31%|███ | 762MB / 2.49GB  ...eckpoints/model_090000.pt: 8%|▊ | 187MB / 2.35GB  Processing Files (0 / 8) : 25%|██▌ | 5.61GB / 22.2GB, 537MB/s New Data Upload : 53%|█████▎ | 1.50GB / 2.82GB, 138MB/s  ...nts/optim_090000_rank1.pt: 32%|███▏ | 798MB / 2.49GB  ...nts/optim_090000_rank2.pt: 31%|███ | 758MB / 2.49GB  ...nts/optim_090000_rank4.pt: 33%|███▎ | 813MB / 2.49GB  ...nts/optim_090000_rank5.pt: 32%|███▏ | 805MB / 2.49GB  ...nts/optim_090000_rank6.pt: 32%|███▏ | 786MB / 2.49GB  ...nts/optim_090000_rank0.pt: 33%|███▎ | 820MB / 2.49GB  ...nts/optim_090000_rank7.pt: 31%|███▏ | 782MB / 2.49GB  ...eckpoints/model_090000.pt: 8%|▊ | 191MB / 2.35GB  Processing Files (0 / 8) : 26%|██▌ | 5.75GB / 22.2GB, 550MB/s New Data Upload : 55%|█████▍ | 1.54GB / 2.82GB, 141MB/s  ...nts/optim_090000_rank1.pt: 33%|███▎ | 818MB / 2.49GB  ...nts/optim_090000_rank2.pt: 31%|███▏ | 780MB / 2.49GB  ...nts/optim_090000_rank4.pt: 34%|███▎ | 836MB / 2.49GB  ...nts/optim_090000_rank5.pt: 33%|███▎ | 828MB / 2.49GB  ...nts/optim_090000_rank6.pt: 32%|███▏ | 808MB / 2.49GB  ...nts/optim_090000_rank0.pt: 34%|███▍ | 846MB / 2.49GB  ...nts/optim_090000_rank7.pt: 32%|███▏ | 804MB / 2.49GB  ...eckpoints/model_090000.pt: 8%|▊ | 195MB / 2.35GB  Processing Files (0 / 8) : 27%|██▋ | 5.91GB / 22.2GB, 564MB/s New Data Upload : 56%|█████▌ | 1.58GB / 2.82GB, 143MB/s  ...nts/optim_090000_rank1.pt: 34%|███▎ | 836MB / 2.49GB  ...nts/optim_090000_rank2.pt: 32%|███▏ | 801MB / 2.49GB  ...nts/optim_090000_rank4.pt: 34%|███▍ | 855MB / 2.49GB  ...nts/optim_090000_rank5.pt: 34%|███▍ | 847MB / 2.49GB  ...nts/optim_090000_rank6.pt: 33%|███▎ | 828MB / 2.49GB  ...nts/optim_090000_rank0.pt: 35%|███▍ | 865MB / 2.49GB  ...nts/optim_090000_rank7.pt: 33%|███▎ | 828MB / 2.49GB  ...eckpoints/model_090000.pt: 9%|▊ | 201MB / 2.35GB  Processing Files (0 / 8) : 27%|██▋ | 6.06GB / 22.2GB, 577MB/s New Data Upload : 56%|█████▋ | 1.62GB / 2.88GB, 146MB/s  ...nts/optim_090000_rank1.pt: 34%|███▍ | 850MB / 2.49GB  ...nts/optim_090000_rank2.pt: 33%|███▎ | 818MB / 2.49GB  ...nts/optim_090000_rank4.pt: 35%|███▌ | 874MB / 2.49GB  ...nts/optim_090000_rank5.pt: 35%|███▍ | 864MB / 2.49GB  ...nts/optim_090000_rank6.pt: 34%|███▍ | 844MB / 2.49GB  ...nts/optim_090000_rank0.pt: 36%|███▌ | 884MB / 2.49GB  ...nts/optim_090000_rank7.pt: 34%|███▍ | 844MB / 2.49GB  ...eckpoints/model_090000.pt: 9%|▉ | 207MB / 2.35GB  Processing Files (0 / 8) : 28%|██▊ | 6.18GB / 22.2GB, 587MB/s New Data Upload : 58%|█████▊ | 1.66GB / 2.88GB, 148MB/s  ...nts/optim_090000_rank1.pt: 35%|███▍ | 869MB / 2.49GB  ...nts/optim_090000_rank2.pt: 34%|███▎ | 837MB / 2.49GB  ...nts/optim_090000_rank4.pt: 36%|███▌ | 891MB / 2.49GB  ...nts/optim_090000_rank5.pt: 35%|███▌ | 881MB / 2.49GB  ...nts/optim_090000_rank6.pt: 35%|███▍ | 863MB / 2.49GB  ...nts/optim_090000_rank0.pt: 36%|███▋ | 903MB / 2.49GB  ...nts/optim_090000_rank7.pt: 35%|███▍ | 864MB / 2.49GB  ...eckpoints/model_090000.pt: 9%|▉ | 214MB / 2.35GB  Processing Files (0 / 8) : 28%|██▊ | 6.32GB / 22.2GB, 598MB/s New Data Upload : 59%|█████▉ | 1.70GB / 2.88GB, 150MB/s  ...nts/optim_090000_rank1.pt: 36%|███▌ | 885MB / 2.49GB  ...nts/optim_090000_rank2.pt: 34%|███▍ | 855MB / 2.49GB  ...nts/optim_090000_rank4.pt: 36%|███▋ | 907MB / 2.49GB  ...nts/optim_090000_rank5.pt: 36%|███▌ | 898MB / 2.49GB  ...nts/optim_090000_rank6.pt: 35%|███▌ | 879MB / 2.49GB  ...nts/optim_090000_rank0.pt: 37%|███▋ | 921MB / 2.49GB  ...nts/optim_090000_rank7.pt: 35%|███▌ | 881MB / 2.49GB  ...eckpoints/model_090000.pt: 9%|▉ | 218MB / 2.35GB  Processing Files (0 / 8) : 29%|██▉ | 6.44GB / 22.2GB, 608MB/s New Data Upload : 59%|█████▉ | 1.74GB / 2.95GB, 152MB/s  ...nts/optim_090000_rank1.pt: 36%|███▋ | 903MB / 2.49GB  ...nts/optim_090000_rank2.pt: 35%|███▌ | 876MB / 2.49GB  ...nts/optim_090000_rank4.pt: 37%|███▋ | 923MB / 2.49GB  ...nts/optim_090000_rank5.pt: 37%|███▋ | 917MB / 2.49GB  ...nts/optim_090000_rank6.pt: 36%|███▌ | 897MB / 2.49GB  ...nts/optim_090000_rank0.pt: 38%|███▊ | 940MB / 2.49GB  ...nts/optim_090000_rank7.pt: 36%|███▌ | 900MB / 2.49GB  ...eckpoints/model_090000.pt: 9%|▉ | 223MB / 2.35GB  Processing Files (0 / 8) : 30%|██▉ | 6.58GB / 22.2GB, 619MB/s New Data Upload : 60%|██████ | 1.77GB / 2.95GB, 153MB/s  ...nts/optim_090000_rank1.pt: 37%|███▋ | 920MB / 2.49GB  ...nts/optim_090000_rank2.pt: 36%|███▌ | 895MB / 2.49GB  ...nts/optim_090000_rank4.pt: 38%|███▊ | 939MB / 2.49GB  ...nts/optim_090000_rank5.pt: 38%|███▊ | 933MB / 2.49GB  ...nts/optim_090000_rank6.pt: 37%|███▋ | 914MB / 2.49GB  ...nts/optim_090000_rank0.pt: 39%|███▊ | 957MB / 2.49GB  ...nts/optim_090000_rank7.pt: 37%|███▋ | 919MB / 2.49GB  ...eckpoints/model_090000.pt: 10%|▉ | 228MB / 2.35GB  Processing Files (0 / 8) : 30%|███ | 6.70GB / 22.2GB, 629MB/s New Data Upload : 60%|█████▉ | 1.81GB / 3.02GB, 154MB/s  ...nts/optim_090000_rank1.pt: 37%|███▋ | 930MB / 2.49GB  ...nts/optim_090000_rank2.pt: 37%|███▋ | 909MB / 2.49GB  ...nts/optim_090000_rank4.pt: 38%|███▊ | 949MB / 2.49GB  ...nts/optim_090000_rank5.pt: 38%|███▊ | 945MB / 2.49GB  ...nts/optim_090000_rank6.pt: 37%|███▋ | 926MB / 2.49GB  ...nts/optim_090000_rank0.pt: 39%|███▉ | 968MB / 2.49GB  ...nts/optim_090000_rank7.pt: 38%|███▊ | 933MB / 2.49GB  ...eckpoints/model_090000.pt: 10%|█ | 235MB / 2.35GB  Processing Files (0 / 8) : 31%|███ | 6.80GB / 22.2GB, 636MB/s New Data Upload : 61%|██████ | 1.84GB / 3.02GB, 156MB/s  ...nts/optim_090000_rank1.pt: 38%|███▊ | 947MB / 2.49GB  ...nts/optim_090000_rank2.pt: 37%|███▋ | 929MB / 2.49GB  ...nts/optim_090000_rank4.pt: 39%|███▉ | 964MB / 2.49GB  ...nts/optim_090000_rank5.pt: 39%|███▊ | 961MB / 2.49GB  ...nts/optim_090000_rank6.pt: 38%|███▊ | 943MB / 2.49GB  ...nts/optim_090000_rank0.pt: 40%|███▉ | 986MB / 2.49GB  ...nts/optim_090000_rank7.pt: 38%|███▊ | 951MB / 2.49GB  ...eckpoints/model_090000.pt: 10%|█ | 240MB / 2.35GB  Processing Files (0 / 8) : 31%|███ | 6.92GB / 22.2GB, 645MB/s New Data Upload : 61%|██████ | 1.87GB / 3.08GB, 157MB/s  ...nts/optim_090000_rank1.pt: 39%|███▉ | 973MB / 2.49GB  ...nts/optim_090000_rank2.pt: 38%|███▊ | 954MB / 2.49GB  ...nts/optim_090000_rank4.pt: 40%|███▉ | 986MB / 2.49GB  ...nts/optim_090000_rank5.pt: 40%|███▉ | 983MB / 2.49GB  ...nts/optim_090000_rank6.pt: 39%|███▉ | 966MB / 2.49GB  ...nts/optim_090000_rank0.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_090000_rank7.pt: 39%|███▉ | 976MB / 2.49GB  ...eckpoints/model_090000.pt: 11%|█ | 247MB / 2.35GB  Processing Files (0 / 8) : 32%|███▏ | 7.10GB / 22.2GB, 660MB/s New Data Upload : 62%|██████▏ | 1.92GB / 3.08GB, 159MB/s  ...nts/optim_090000_rank1.pt: 40%|███▉ | 990MB / 2.49GB  ...nts/optim_090000_rank2.pt: 39%|███▉ | 972MB / 2.49GB  ...nts/optim_090000_rank4.pt: 40%|████ | 1.00GB / 2.49GB  ...nts/optim_090000_rank5.pt: 40%|████ | 998MB / 2.49GB  ...nts/optim_090000_rank6.pt: 40%|███▉ | 984MB / 2.49GB  ...nts/optim_090000_rank0.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_090000_rank7.pt: 40%|████ | 996MB / 2.49GB  ...eckpoints/model_090000.pt: 11%|█ | 251MB / 2.35GB  Processing Files (0 / 8) : 32%|███▏ | 7.22GB / 22.2GB, 669MB/s New Data Upload : 63%|██████▎ | 1.95GB / 3.08GB, 160MB/s  ...nts/optim_090000_rank1.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_090000_rank2.pt: 40%|███▉ | 992MB / 2.49GB  ...nts/optim_090000_rank4.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_090000_rank5.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_090000_rank6.pt: 40%|████ | 1.00GB / 2.49GB  ...nts/optim_090000_rank0.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_090000_rank7.pt: 41%|████ | 1.02GB / 2.49GB  ...eckpoints/model_090000.pt: 11%|█ | 261MB / 2.35GB  Processing Files (0 / 8) : 33%|███▎ | 7.37GB / 22.2GB, 682MB/s New Data Upload : 63%|██████▎ | 2.00GB / 3.15GB, 162MB/s  ...nts/optim_090000_rank1.pt: 42%|████▏ | 1.03GB / 2.49GB  ...nts/optim_090000_rank2.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_090000_rank4.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_090000_rank5.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_090000_rank6.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_090000_rank0.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_090000_rank7.pt: 42%|████▏ | 1.04GB / 2.49GB  ...eckpoints/model_090000.pt: 11%|█ | 264MB / 2.35GB  Processing Files (0 / 8) : 34%|███▍ | 7.55GB / 22.2GB, 697MB/s New Data Upload : 62%|██████▏ | 2.04GB / 3.29GB, 164MB/s  ...nts/optim_090000_rank1.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_090000_rank2.pt: 42%|████▏ | 1.04GB / 2.49GB  ...nts/optim_090000_rank4.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_090000_rank5.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_090000_rank6.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_090000_rank0.pt: 44%|████▍ | 1.09GB / 2.49GB  ...nts/optim_090000_rank7.pt: 43%|████▎ | 1.06GB / 2.49GB  ...eckpoints/model_090000.pt: 11%|█▏ | 269MB / 2.35GB  Processing Files (0 / 8) : 35%|███▍ | 7.68GB / 22.2GB, 706MB/s New Data Upload : 63%|██████▎ | 2.07GB / 3.29GB, 166MB/s  ...nts/optim_090000_rank1.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_090000_rank2.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_090000_rank4.pt: 44%|████▎ | 1.08GB / 2.49GB  ...nts/optim_090000_rank5.pt: 43%|████▎ | 1.08GB / 2.49GB  ...nts/optim_090000_rank6.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_090000_rank0.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_090000_rank7.pt: 44%|████▎ | 1.08GB / 2.49GB  ...eckpoints/model_090000.pt: 12%|█▏ | 274MB / 2.35GB  Processing Files (0 / 8) : 35%|███▌ | 7.83GB / 22.2GB, 717MB/s New Data Upload : 63%|██████▎ | 2.10GB / 3.35GB, 167MB/s  ...nts/optim_090000_rank1.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_090000_rank2.pt: 44%|████▍ | 1.09GB / 2.49GB  ...nts/optim_090000_rank4.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_090000_rank5.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_090000_rank6.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_090000_rank0.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_090000_rank7.pt: 45%|████▍ | 1.11GB / 2.49GB  ...eckpoints/model_090000.pt: 12%|█▏ | 280MB / 2.35GB  Processing Files (0 / 8) : 36%|███▌ | 8.05GB / 22.2GB, 735MB/s New Data Upload : 63%|██████▎ | 2.14GB / 3.42GB, 170MB/s  ...nts/optim_090000_rank1.pt: 46%|████▌ | 1.13GB / 2.49GB  ...nts/optim_090000_rank2.pt: 45%|████▌ | 1.12GB / 2.49GB  ...nts/optim_090000_rank4.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_090000_rank5.pt: 46%|████▌ | 1.13GB / 2.49GB  ...nts/optim_090000_rank6.pt: 45%|████▌ | 1.13GB / 2.49GB  ...nts/optim_090000_rank0.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_090000_rank7.pt: 46%|████▌ | 1.14GB / 2.49GB  ...eckpoints/model_090000.pt: 12%|█▏ | 286MB / 2.35GB  Processing Files (0 / 8) : 37%|███▋ | 8.24GB / 22.2GB, 745MB/s New Data Upload : 63%|██████▎ | 2.18GB / 3.49GB, 171MB/s  ...nts/optim_090000_rank1.pt: 46%|████▌ | 1.15GB / 2.49GB  ...nts/optim_090000_rank2.pt: 46%|████▌ | 1.13GB / 2.49GB  ...nts/optim_090000_rank4.pt: 46%|████▋ | 1.15GB / 2.49GB  ...nts/optim_090000_rank5.pt: 46%|████▌ | 1.15GB / 2.49GB  ...nts/optim_090000_rank6.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_090000_rank0.pt: 47%|████▋ | 1.18GB / 2.49GB  ...nts/optim_090000_rank7.pt: 46%|████▋ | 1.15GB / 2.49GB  ...eckpoints/model_090000.pt: 12%|█▏ | 291MB / 2.35GB  Processing Files (0 / 8) : 38%|███▊ | 8.35GB / 22.2GB, 748MB/s New Data Upload : 62%|██████▏ | 2.21GB / 3.55GB, 171MB/s  ...nts/optim_090000_rank1.pt: 48%|████▊ | 1.18GB / 2.49GB  ...nts/optim_090000_rank2.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_090000_rank4.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_090000_rank5.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_090000_rank6.pt: 47%|████▋ | 1.18GB / 2.49GB  ...nts/optim_090000_rank0.pt: 49%|████▉ | 1.21GB / 2.49GB  ...nts/optim_090000_rank7.pt: 48%|████▊ | 1.19GB / 2.49GB  ...eckpoints/model_090000.pt: 13%|█▎ | 300MB / 2.35GB  Processing Files (0 / 8) : 39%|███▊ | 8.60GB / 22.2GB, 761MB/s New Data Upload : 62%|██████▏ | 2.26GB / 3.62GB, 174MB/s  ...nts/optim_090000_rank1.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_090000_rank2.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_090000_rank4.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_090000_rank5.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_090000_rank6.pt: 49%|████▊ | 1.21GB / 2.49GB  ...nts/optim_090000_rank0.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_090000_rank7.pt: 49%|████▉ | 1.22GB / 2.49GB  ...eckpoints/model_090000.pt: 13%|█▎ | 305MB / 2.35GB  Processing Files (0 / 8) : 40%|███▉ | 8.83GB / 22.2GB, 772MB/s New Data Upload : 62%|██████▏ | 2.30GB / 3.69GB, 176MB/s  ...nts/optim_090000_rank1.pt: 50%|█████ | 1.24GB / 2.49GB  ...nts/optim_090000_rank2.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_090000_rank4.pt: 50%|█████ | 1.25GB / 2.49GB  ...nts/optim_090000_rank5.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_090000_rank6.pt: 50%|████▉ | 1.23GB / 2.49GB  ...nts/optim_090000_rank0.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_090000_rank7.pt: 50%|████▉ | 1.24GB / 2.49GB  ...eckpoints/model_090000.pt: 13%|█▎ | 314MB / 2.35GB  Processing Files (0 / 8) : 41%|████ | 9.02GB / 22.2GB, 777MB/s New Data Upload : 64%|██████▎ | 2.35GB / 3.69GB, 176MB/s  ...nts/optim_090000_rank1.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_090000_rank2.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_090000_rank4.pt: 52%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_090000_rank5.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_090000_rank6.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_090000_rank0.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_090000_rank7.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...eckpoints/model_090000.pt: 14%|█▎ | 321MB / 2.35GB  Processing Files (0 / 8) : 42%|████▏ | 9.27GB / 22.2GB, 788MB/s New Data Upload : 65%|██████▌ | 2.40GB / 3.69GB, 178MB/s  ...nts/optim_090000_rank1.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_090000_rank2.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_090000_rank4.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_090000_rank5.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_090000_rank6.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_090000_rank0.pt: 54%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_090000_rank7.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...eckpoints/model_090000.pt: 14%|█▍ | 327MB / 2.35GB  Processing Files (0 / 8) : 43%|████▎ | 9.47GB / 22.2GB, 794MB/s New Data Upload : 64%|██████▍ | 2.44GB / 3.82GB, 179MB/s  ...nts/optim_090000_rank1.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_090000_rank2.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_090000_rank4.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_090000_rank5.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_090000_rank6.pt: 53%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_090000_rank0.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_090000_rank7.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...eckpoints/model_090000.pt: 14%|█▍ | 331MB / 2.35GB  Processing Files (0 / 8) : 44%|████▎ | 9.72GB / 22.2GB, 799MB/s New Data Upload : 62%|██████▏ | 2.49GB / 4.02GB, 180MB/s  ...nts/optim_090000_rank1.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_090000_rank2.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_090000_rank4.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_090000_rank5.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_090000_rank6.pt: 55%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_090000_rank0.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_090000_rank7.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...eckpoints/model_090000.pt: 14%|█▍ | 338MB / 2.35GB  Processing Files (0 / 8) : 45%|████▍ | 9.91GB / 22.2GB, 802MB/s New Data Upload : 61%|██████ | 2.53GB / 4.16GB, 180MB/s  ...nts/optim_090000_rank1.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_090000_rank2.pt: 55%|█████▌ | 1.37GB / 2.49GB  ...nts/optim_090000_rank4.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_090000_rank5.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_090000_rank6.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_090000_rank0.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_090000_rank7.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...eckpoints/model_090000.pt: 15%|█▍ | 343MB / 2.35GB  Processing Files (0 / 8) : 45%|████▌ | 10.1GB / 22.2GB, 807MB/s New Data Upload : 62%|██████▏ | 2.56GB / 4.16GB, 181MB/s  ...nts/optim_090000_rank1.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_090000_rank2.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...nts/optim_090000_rank4.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_090000_rank5.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_090000_rank6.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...nts/optim_090000_rank0.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_090000_rank7.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...eckpoints/model_090000.pt: 15%|█▌ | 353MB / 2.35GB  Processing Files (0 / 8) : 46%|████▋ | 10.3GB / 22.2GB, 812MB/s New Data Upload : 62%|██████▏ | 2.61GB / 4.22GB, 182MB/s  ...nts/optim_090000_rank1.pt: 59%|█████▉ | 1.46GB / 2.49GB  ...nts/optim_090000_rank2.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_090000_rank4.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_090000_rank5.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_090000_rank6.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_090000_rank0.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_090000_rank7.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...eckpoints/model_090000.pt: 15%|█▌ | 354MB / 2.35GB  Processing Files (0 / 8) : 48%|████▊ | 10.6GB / 22.2GB, 821MB/s New Data Upload : 63%|██████▎ | 2.66GB / 4.22GB, 183MB/s  ...nts/optim_090000_rank1.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 59%|█████▉ | 1.46GB / 2.49GB  ...nts/optim_090000_rank4.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_090000_rank5.pt: 59%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_090000_rank6.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_090000_rank0.pt: 61%|██████ | 1.50GB / 2.49GB  ...nts/optim_090000_rank7.pt: 59%|█████▉ | 1.48GB / 2.49GB  ...eckpoints/model_090000.pt: 15%|█▌ | 362MB / 2.35GB  Processing Files (0 / 8) : 48%|████▊ | 10.7GB / 22.2GB, 818MB/s New Data Upload : 64%|██████▍ | 2.69GB / 4.22GB, 183MB/s  ...nts/optim_090000_rank1.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_090000_rank2.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_090000_rank5.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_090000_rank6.pt: 60%|██████ | 1.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_090000_rank7.pt: 60%|██████ | 1.50GB / 2.49GB  ...eckpoints/model_090000.pt: 16%|█▌ | 367MB / 2.35GB  Processing Files (0 / 8) : 49%|████▉ | 10.9GB / 22.2GB, 821MB/s New Data Upload : 64%|██████▎ | 2.73GB / 4.29GB, 184MB/s  ...nts/optim_090000_rank1.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_090000_rank2.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_090000_rank4.pt: 61%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_090000_rank5.pt: 61%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_090000_rank6.pt: 61%|██████ | 1.52GB / 2.49GB  ...nts/optim_090000_rank0.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_090000_rank7.pt: 61%|██████▏ | 1.53GB / 2.49GB  ...eckpoints/model_090000.pt: 16%|█▌ | 372MB / 2.35GB  Processing Files (0 / 8) : 50%|████▉ | 11.1GB / 22.2GB, 822MB/s New Data Upload : 65%|██████▍ | 2.77GB / 4.29GB, 185MB/s  ...nts/optim_090000_rank1.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_090000_rank2.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_090000_rank4.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_090000_rank5.pt: 63%|██████▎ | 1.55GB / 2.49GB  ...nts/optim_090000_rank6.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_090000_rank0.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_090000_rank7.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...eckpoints/model_090000.pt: 16%|█▌ | 378MB / 2.35GB  Processing Files (0 / 8) : 51%|█████ | 11.3GB / 22.2GB, 825MB/s New Data Upload : 65%|██████▍ | 2.82GB / 4.36GB, 187MB/s  ...nts/optim_090000_rank1.pt: 64%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_090000_rank2.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_090000_rank4.pt: 63%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_090000_rank5.pt: 63%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_090000_rank6.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_090000_rank0.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_090000_rank7.pt: 63%|██████▎ | 1.58GB / 2.49GB  ...eckpoints/model_090000.pt: 16%|█▋ | 382MB / 2.35GB  Processing Files (0 / 8) : 51%|█████▏ | 11.4GB / 22.2GB, 822MB/s New Data Upload : 64%|██████▎ | 2.86GB / 4.49GB, 187MB/s  ...nts/optim_090000_rank1.pt: 65%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_090000_rank2.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_090000_rank4.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_090000_rank5.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_090000_rank6.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_090000_rank0.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_090000_rank7.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...eckpoints/model_090000.pt: 16%|█▋ | 387MB / 2.35GB  Processing Files (0 / 8) : 52%|█████▏ | 11.6GB / 22.2GB, 823MB/s New Data Upload : 64%|██████▎ | 2.91GB / 4.56GB, 189MB/s  ...nts/optim_090000_rank1.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_090000_rank2.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_090000_rank4.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_090000_rank5.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_090000_rank6.pt: 65%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_090000_rank0.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_090000_rank7.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...eckpoints/model_090000.pt: 17%|█▋ | 392MB / 2.35GB  Processing Files (0 / 8) : 53%|█████▎ | 11.8GB / 22.2GB, 833MB/s New Data Upload : 65%|██████▍ | 2.96GB / 4.56GB, 191MB/s  ...nts/optim_090000_rank3.pt: 1%| | 15.4MB / 2.49GB  ...nts/optim_090000_rank1.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_090000_rank2.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_090000_rank4.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_090000_rank5.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_090000_rank6.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_090000_rank0.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_090000_rank7.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...eckpoints/model_090000.pt: 17%|█▋ | 405MB / 2.35GB  ...nts/optim_090000_rank3.pt: 1%| | 15.4MB / 2.49GB  Processing Files (0 / 9) : 54%|█████▍ | 12.0GB / 22.2GB, 832MB/s New Data Upload : 64%|██████▍ | 3.00GB / 4.70GB, 192MB/s  ...nts/optim_090000_rank1.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_090000_rank2.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_090000_rank4.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_090000_rank5.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_090000_rank6.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_090000_rank0.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_090000_rank7.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...eckpoints/model_090000.pt: 17%|█▋ | 410MB / 2.35GB  ...nts/optim_090000_rank3.pt: 6%|▌ | 155MB / 2.49GB  Processing Files (0 / 9) : 55%|█████▌ | 12.3GB / 22.2GB, 847MB/s New Data Upload : 64%|██████▍ | 3.04GB / 4.77GB, 193MB/s  ...nts/optim_090000_rank1.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_090000_rank2.pt: 68%|██████▊ | 1.68GB / 2.49GB  ...nts/optim_090000_rank4.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_090000_rank5.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_090000_rank6.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_090000_rank0.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_090000_rank7.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...eckpoints/model_090000.pt: 18%|█▊ | 417MB / 2.35GB  ...nts/optim_090000_rank3.pt: 12%|█▏ | 286MB / 2.49GB  Processing Files (0 / 9) : 57%|█████▋ | 12.6GB / 22.2GB, 864MB/s New Data Upload : 65%|██████▍ | 3.09GB / 4.77GB, 195MB/s  ...nts/optim_090000_rank1.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_090000_rank2.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_090000_rank4.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_090000_rank5.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_090000_rank6.pt: 69%|██████▊ | 1.71GB / 2.49GB  ...nts/optim_090000_rank0.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_090000_rank7.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...eckpoints/model_090000.pt: 18%|█▊ | 425MB / 2.35GB  ...nts/optim_090000_rank3.pt: 17%|█▋ | 430MB / 2.49GB  Processing Files (0 / 9) : 58%|█████▊ | 12.9GB / 22.2GB, 872MB/s New Data Upload : 65%|██████▌ | 3.12GB / 4.77GB, 195MB/s  ...nts/optim_090000_rank1.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_090000_rank2.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_090000_rank4.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_090000_rank5.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_090000_rank6.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...nts/optim_090000_rank0.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_090000_rank7.pt: 70%|███████ | 1.74GB / 2.49GB  ...eckpoints/model_090000.pt: 18%|█▊ | 430MB / 2.35GB  ...nts/optim_090000_rank3.pt: 23%|██▎ | 562MB / 2.49GB  Processing Files (0 / 9) : 59%|█████▉ | 13.2GB / 22.2GB, 884MB/s New Data Upload : 66%|██████▋ | 3.17GB / 4.77GB, 196MB/s  ...nts/optim_090000_rank1.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_090000_rank2.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_090000_rank4.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_090000_rank5.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_090000_rank6.pt: 71%|███████ | 1.75GB / 2.49GB  ...nts/optim_090000_rank0.pt: 73%|███████▎ | 1.80GB / 2.49GB  ...nts/optim_090000_rank7.pt: 71%|███████ | 1.77GB / 2.49GB  ...eckpoints/model_090000.pt: 19%|█▊ | 438MB / 2.35GB  ...nts/optim_090000_rank3.pt: 27%|██▋ | 672MB / 2.49GB  Processing Files (0 / 9) : 61%|██████ | 13.5GB / 22.2GB, 898MB/s New Data Upload : 67%|██████▋ | 3.22GB / 4.77GB, 197MB/s  ...nts/optim_090000_rank1.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_090000_rank2.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_090000_rank4.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_090000_rank5.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_090000_rank6.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_090000_rank0.pt: 74%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_090000_rank7.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...eckpoints/model_090000.pt: 19%|█▉ | 448MB / 2.35GB  ...nts/optim_090000_rank3.pt: 31%|███▏ | 779MB / 2.49GB  Processing Files (0 / 9) : 62%|██████▏ | 13.8GB / 22.2GB, 912MB/s New Data Upload : 68%|██████▊ | 3.26GB / 4.77GB, 199MB/s  ...nts/optim_090000_rank1.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_090000_rank2.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_090000_rank4.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_090000_rank5.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_090000_rank6.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_090000_rank0.pt: 75%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_090000_rank7.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...eckpoints/model_090000.pt: 19%|█▉ | 452MB / 2.35GB  ...nts/optim_090000_rank3.pt: 36%|███▌ | 895MB / 2.49GB  Processing Files (0 / 9) : 63%|██████▎ | 14.1GB / 22.2GB, 923MB/s New Data Upload : 69%|██████▉ | 3.31GB / 4.77GB, 200MB/s  ...nts/optim_090000_rank1.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_090000_rank2.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_090000_rank4.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_090000_rank5.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_090000_rank6.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_090000_rank0.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_090000_rank7.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...eckpoints/model_090000.pt: 20%|█▉ | 459MB / 2.35GB  ...nts/optim_090000_rank3.pt: 40%|████ | 999MB / 2.49GB  Processing Files (0 / 9) : 65%|██████▍ | 14.4GB / 22.2GB, 938MB/s New Data Upload : 71%|███████ | 3.36GB / 4.77GB, 202MB/s  ...nts/optim_090000_rank1.pt: 75%|███████▍ | 1.86GB / 2.49GB [A ...nts/optim_090000_rank2.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_090000_rank4.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_090000_rank5.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_090000_rank6.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_090000_rank0.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_090000_rank7.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...eckpoints/model_090000.pt: 20%|█▉ | 466MB / 2.35GB  ...nts/optim_090000_rank3.pt: 43%|████▎ | 1.07GB / 2.49GB  Processing Files (0 / 9) : 66%|██████▌ | 14.6GB / 22.2GB, 944MB/s New Data Upload : 71%|███████▏ | 3.41GB / 4.77GB, 203MB/s  ...nts/optim_090000_rank1.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_090000_rank2.pt: 75%|███████▌ | 1.86GB / 2.49GB  ...nts/optim_090000_rank4.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_090000_rank5.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_090000_rank6.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_090000_rank0.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_090000_rank7.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...eckpoints/model_090000.pt: 20%|██ | 472MB / 2.35GB  ...nts/optim_090000_rank3.pt: 49%|████▊ | 1.21GB / 2.49GB  Processing Files (0 / 9) : 67%|██████▋ | 14.9GB / 22.2GB, 960MB/s New Data Upload : 71%|███████▏ | 3.45GB / 4.84GB, 203MB/s  ...nts/optim_090000_rank1.pt: 77%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_090000_rank2.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_090000_rank4.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_090000_rank5.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_090000_rank6.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_090000_rank0.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_090000_rank7.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...eckpoints/model_090000.pt: 21%|██ | 483MB / 2.35GB  ...nts/optim_090000_rank3.pt: 51%|█████▏ | 1.28GB / 2.49GB  Processing Files (0 / 9) : 68%|██████▊ | 15.1GB / 22.2GB, 968MB/s New Data Upload : 71%|███████▏ | 3.50GB / 4.90GB, 205MB/s  ...nts/optim_090000_rank1.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_090000_rank2.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_090000_rank4.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_090000_rank5.pt: 77%|███████▋ | 1.93GB / 2.49GB  ...nts/optim_090000_rank6.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_090000_rank0.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_090000_rank7.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...eckpoints/model_090000.pt: 21%|██ | 490MB / 2.35GB  ...nts/optim_090000_rank3.pt: 59%|█████▉ | 1.48GB / 2.49GB  Processing Files (0 / 9) : 70%|██████▉ | 15.5GB / 22.2GB, 1.00GB/s New Data Upload : 72%|███████▏ | 3.55GB / 4.90GB, 208MB/s  ...nts/optim_090000_rank1.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_090000_rank2.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_090000_rank4.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_090000_rank5.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_090000_rank6.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_090000_rank0.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_090000_rank7.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...eckpoints/model_090000.pt: 21%|██ | 498MB / 2.35GB  ...nts/optim_090000_rank3.pt: 61%|██████ | 1.52GB / 2.49GB  Processing Files (0 / 9) : 71%|███████ | 15.7GB / 22.2GB, 1.00GB/s New Data Upload : 73%|███████▎ | 3.59GB / 4.90GB, 208MB/s  ...nts/optim_090000_rank1.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_090000_rank2.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_090000_rank4.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_090000_rank5.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_090000_rank6.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_090000_rank0.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_090000_rank7.pt: 80%|███████▉ | 1.99GB / 2.49GB  ...eckpoints/model_090000.pt: 22%|██▏ | 510MB / 2.35GB  ...nts/optim_090000_rank3.pt: 63%|██████▎ | 1.57GB / 2.49GB  Processing Files (0 / 9) : 71%|███████▏ | 15.9GB / 22.2GB, 1.01GB/s New Data Upload : 73%|███████▎ | 3.64GB / 4.97GB, 209MB/s  ...nts/optim_090000_rank1.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_090000_rank2.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_090000_rank4.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_090000_rank5.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_090000_rank6.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_090000_rank0.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_090000_rank7.pt: 81%|████████ | 2.01GB / 2.49GB  ...eckpoints/model_090000.pt: 22%|██▏ | 525MB / 2.35GB  ...nts/optim_090000_rank3.pt: 64%|██████▍ | 1.60GB / 2.49GB  Processing Files (0 / 9) : 72%|███████▏ | 16.0GB / 22.2GB, 1.01GB/s New Data Upload : 74%|███████▍ | 3.68GB / 4.97GB, 210MB/s  ...nts/optim_090000_rank1.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_090000_rank2.pt: 80%|███████▉ | 1.99GB / 2.49GB  ...nts/optim_090000_rank4.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_090000_rank5.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_090000_rank6.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_090000_rank0.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_090000_rank7.pt: 81%|████████▏ | 2.03GB / 2.49GB  ...eckpoints/model_090000.pt: 23%|██▎ | 539MB / 2.35GB  ...nts/optim_090000_rank3.pt: 67%|██████▋ | 1.66GB / 2.49GB  Processing Files (0 / 9) : 73%|███████▎ | 16.2GB / 22.2GB, 1.01GB/s New Data Upload : 75%|███████▌ | 3.73GB / 4.97GB, 211MB/s  ...nts/optim_090000_rank1.pt: 81%|████████ | 2.02GB / 2.49GB  ...nts/optim_090000_rank2.pt: 81%|████████ | 2.00GB / 2.49GB  ...nts/optim_090000_rank4.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_090000_rank5.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_090000_rank6.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_090000_rank0.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_090000_rank7.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...eckpoints/model_090000.pt: 23%|██▎ | 552MB / 2.35GB  ...nts/optim_090000_rank3.pt: 73%|███████▎ | 1.82GB / 2.49GB  Processing Files (0 / 9) : 74%|███████▍ | 16.5GB / 22.2GB, 1.03GB/s New Data Upload : 76%|███████▌ | 3.77GB / 4.97GB, 210MB/s  ...nts/optim_090000_rank1.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_090000_rank2.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_090000_rank4.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_090000_rank5.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_090000_rank6.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_090000_rank0.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_090000_rank7.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...eckpoints/model_090000.pt: 24%|██▍ | 571MB / 2.35GB  ...nts/optim_090000_rank3.pt: 75%|███████▌ | 1.87GB / 2.49GB  Processing Files (0 / 9) : 75%|███████▌ | 16.7GB / 22.2GB, 1.03GB/s New Data Upload : 76%|███████▌ | 3.82GB / 5.04GB, 211MB/s  ...nts/optim_090000_rank1.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_090000_rank2.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_090000_rank4.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_090000_rank5.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_090000_rank6.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_090000_rank0.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_090000_rank7.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...eckpoints/model_090000.pt: 25%|██▌ | 590MB / 2.35GB  ...nts/optim_090000_rank3.pt: 81%|████████ | 2.01GB / 2.49GB  Processing Files (0 / 9) : 76%|███████▋ | 17.0GB / 22.2GB, 1.04GB/s New Data Upload : 77%|███████▋ | 3.86GB / 5.04GB, 212MB/s  ...nts/optim_090000_rank1.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_090000_rank2.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_090000_rank4.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_090000_rank5.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_090000_rank6.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_090000_rank0.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_090000_rank7.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...eckpoints/model_090000.pt: 26%|██▌ | 611MB / 2.35GB  ...nts/optim_090000_rank3.pt: 81%|████████▏ | 2.02GB / 2.49GB  Processing Files (0 / 9) : 77%|███████▋ | 17.1GB / 22.2GB, 1.05GB/s New Data Upload : 77%|███████▋ | 3.91GB / 5.10GB, 213MB/s  ...nts/optim_090000_rank1.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_090000_rank2.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_090000_rank4.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_090000_rank5.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_090000_rank6.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_090000_rank0.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_090000_rank7.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...eckpoints/model_090000.pt: 27%|██▋ | 630MB / 2.35GB  ...nts/optim_090000_rank3.pt: 82%|████████▏ | 2.05GB / 2.49GB  Processing Files (0 / 9) : 78%|███████▊ | 17.3GB / 22.2GB, 1.05GB/s New Data Upload : 77%|███████▋ | 3.96GB / 5.10GB, 214MB/s  ...nts/optim_090000_rank1.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_090000_rank2.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_090000_rank4.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_090000_rank5.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_090000_rank6.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_090000_rank0.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_090000_rank7.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...eckpoints/model_090000.pt: 27%|██▋ | 644MB / 2.35GB  ...nts/optim_090000_rank3.pt: 83%|████████▎ | 2.06GB / 2.49GB  Processing Files (0 / 9) : 78%|███████▊ | 17.4GB / 22.2GB, 1.05GB/s New Data Upload : 77%|███████▋ | 3.99GB / 5.17GB, 214MB/s  ...nts/optim_090000_rank1.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_090000_rank2.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_090000_rank4.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_090000_rank5.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_090000_rank6.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_090000_rank0.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_090000_rank7.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...eckpoints/model_090000.pt: 28%|██▊ | 665MB / 2.35GB  ...nts/optim_090000_rank3.pt: 84%|████████▎ | 2.08GB / 2.49GB  Processing Files (0 / 9) : 79%|███████▉ | 17.6GB / 22.2GB, 1.06GB/s New Data Upload : 78%|███████▊ | 4.04GB / 5.17GB, 216MB/s  ...nts/optim_090000_rank1.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_090000_rank2.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_090000_rank4.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_090000_rank5.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_090000_rank6.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_090000_rank0.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_090000_rank7.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...eckpoints/model_090000.pt: 29%|██▉ | 680MB / 2.35GB  ...nts/optim_090000_rank3.pt: 84%|████████▍ | 2.09GB / 2.49GB  Processing Files (0 / 9) : 79%|███████▉ | 17.7GB / 22.2GB, 1.05GB/s New Data Upload : 79%|███████▉ | 4.07GB / 5.17GB, 216MB/s  ...nts/optim_090000_rank1.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_090000_rank2.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_090000_rank4.pt: 86%|████████▋ | 2.14GB / 2.49GB  ...nts/optim_090000_rank5.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_090000_rank6.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_090000_rank0.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_090000_rank7.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...eckpoints/model_090000.pt: 30%|██▉ | 699MB / 2.35GB  ...nts/optim_090000_rank3.pt: 85%|████████▍ | 2.10GB / 2.49GB  Processing Files (0 / 9) : 80%|███████▉ | 17.8GB / 22.2GB, 1.05GB/s New Data Upload : 78%|███████▊ | 4.11GB / 5.24GB, 215MB/s  ...nts/optim_090000_rank1.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_090000_rank2.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_090000_rank4.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_090000_rank5.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_090000_rank6.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_090000_rank0.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_090000_rank7.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...eckpoints/model_090000.pt: 31%|███ | 717MB / 2.35GB  ...nts/optim_090000_rank3.pt: 85%|████████▌ | 2.11GB / 2.49GB  Processing Files (0 / 9) : 80%|████████ | 17.9GB / 22.2GB, 1.05GB/s New Data Upload : 79%|███████▉ | 4.15GB / 5.24GB, 216MB/s  ...nts/optim_090000_rank1.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_090000_rank2.pt: 86%|████████▋ | 2.14GB / 2.49GB  ...nts/optim_090000_rank4.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_090000_rank5.pt: 86%|████████▋ | 2.14GB / 2.49GB  ...nts/optim_090000_rank6.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_090000_rank0.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_090000_rank7.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...eckpoints/model_090000.pt: 31%|███ | 733MB / 2.35GB  ...nts/optim_090000_rank3.pt: 85%|████████▌ | 2.12GB / 2.49GB  Processing Files (0 / 9) : 81%|████████ | 18.0GB / 22.2GB, 1.04GB/s New Data Upload : 79%|███████▉ | 4.18GB / 5.31GB, 215MB/s  ...nts/optim_090000_rank1.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_090000_rank2.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_090000_rank4.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_090000_rank5.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_090000_rank6.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_090000_rank0.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_090000_rank7.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...eckpoints/model_090000.pt: 32%|███▏ | 755MB / 2.35GB  ...nts/optim_090000_rank3.pt: 86%|████████▌ | 2.13GB / 2.49GB  Processing Files (0 / 9) : 81%|████████ | 18.1GB / 22.2GB, 1.03GB/s New Data Upload : 80%|███████▉ | 4.22GB / 5.31GB, 215MB/s  ...nts/optim_090000_rank1.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_090000_rank2.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_090000_rank4.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_090000_rank5.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_090000_rank6.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_090000_rank0.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_090000_rank7.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...eckpoints/model_090000.pt: 33%|███▎ | 771MB / 2.35GB  ...nts/optim_090000_rank3.pt: 86%|████████▌ | 2.14GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.1GB / 22.2GB, 1.02GB/s New Data Upload : 79%|███████▉ | 4.26GB / 5.37GB, 215MB/s  ...nts/optim_090000_rank1.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_090000_rank2.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_090000_rank4.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_090000_rank5.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_090000_rank6.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_090000_rank0.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_090000_rank7.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...eckpoints/model_090000.pt: 34%|███▍ | 793MB / 2.35GB  ...nts/optim_090000_rank3.pt: 87%|████████▋ | 2.15GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.2GB / 22.2GB, 1.02GB/s New Data Upload : 79%|███████▉ | 4.30GB / 5.44GB, 216MB/s  ...nts/optim_090000_rank1.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_090000_rank2.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_090000_rank4.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_090000_rank5.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_090000_rank6.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_090000_rank0.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_090000_rank7.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...eckpoints/model_090000.pt: 35%|███▍ | 814MB / 2.35GB  ...nts/optim_090000_rank3.pt: 87%|████████▋ | 2.16GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.3GB / 22.2GB, 1.00GB/s New Data Upload : 80%|███████▉ | 4.33GB / 5.44GB, 215MB/s  ...nts/optim_090000_rank1.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_090000_rank2.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_090000_rank4.pt: 89%|████████▊ | 2.21GB / 2.49GB  ...nts/optim_090000_rank5.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_090000_rank6.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_090000_rank0.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_090000_rank7.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...eckpoints/model_090000.pt: 36%|███▌ | 838MB / 2.35GB  ...nts/optim_090000_rank3.pt: 87%|████████▋ | 2.17GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.4GB / 22.2GB, 999MB/s New Data Upload : 81%|████████ | 4.38GB / 5.44GB, 216MB/s  ...nts/optim_090000_rank1.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_090000_rank2.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_090000_rank4.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_090000_rank5.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_090000_rank6.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_090000_rank0.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_090000_rank7.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...eckpoints/model_090000.pt: 36%|███▋ | 852MB / 2.35GB  ...nts/optim_090000_rank3.pt: 88%|████████▊ | 2.18GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.5GB / 22.2GB, 993MB/s New Data Upload : 80%|███████▉ | 4.40GB / 5.51GB, 215MB/s  ...nts/optim_090000_rank1.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_090000_rank2.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_090000_rank4.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_090000_rank5.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_090000_rank6.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_090000_rank0.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_090000_rank7.pt: 90%|█████████ | 2.24GB / 2.49GB  ...eckpoints/model_090000.pt: 37%|███▋ | 875MB / 2.35GB  ...nts/optim_090000_rank3.pt: 88%|████████▊ | 2.19GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▎ | 18.6GB / 22.2GB, 977MB/s New Data Upload : 81%|████████ | 4.44GB / 5.51GB, 214MB/s  ...nts/optim_090000_rank1.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_090000_rank2.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_090000_rank4.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_090000_rank5.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_090000_rank6.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_090000_rank0.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_090000_rank7.pt: 90%|█████████ | 2.25GB / 2.49GB  ...eckpoints/model_090000.pt: 38%|███▊ | 895MB / 2.35GB  ...nts/optim_090000_rank3.pt: 89%|████████▊ | 2.20GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▍ | 18.7GB / 22.2GB, 965MB/s New Data Upload : 80%|████████ | 4.48GB / 5.57GB, 214MB/s  ...nts/optim_090000_rank1.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_090000_rank2.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_090000_rank4.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_090000_rank5.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_090000_rank6.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_090000_rank0.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_090000_rank7.pt: 91%|█████████ | 2.25GB / 2.49GB  ...eckpoints/model_090000.pt: 39%|███▉ | 917MB / 2.35GB  ...nts/optim_090000_rank3.pt: 89%|████████▉ | 2.21GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▍ | 18.7GB / 22.2GB, 954MB/s New Data Upload : 80%|████████ | 4.52GB / 5.64GB, 213MB/s  ...nts/optim_090000_rank1.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_090000_rank2.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_090000_rank4.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_090000_rank5.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_090000_rank6.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_090000_rank0.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_090000_rank7.pt: 91%|█████████ | 2.26GB / 2.49GB  ...eckpoints/model_090000.pt: 40%|███▉ | 934MB / 2.35GB  ...nts/optim_090000_rank3.pt: 89%|████████▉ | 2.22GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▍ | 18.8GB / 22.2GB, 937MB/s New Data Upload : 80%|███████▉ | 4.55GB / 5.71GB, 211MB/s  ...nts/optim_090000_rank1.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_090000_rank2.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_090000_rank4.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_090000_rank5.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_090000_rank6.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_090000_rank0.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_090000_rank7.pt: 91%|█████████ | 2.26GB / 2.49GB  ...eckpoints/model_090000.pt: 41%|████ | 962MB / 2.35GB  ...nts/optim_090000_rank3.pt: 89%|████████▉ | 2.22GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▌ | 18.9GB / 22.2GB, 925MB/s New Data Upload : 80%|████████ | 4.59GB / 5.71GB, 211MB/s  ...nts/optim_090000_rank1.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_090000_rank2.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_090000_rank4.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_090000_rank5.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_090000_rank6.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_090000_rank0.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_090000_rank7.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...eckpoints/model_090000.pt: 42%|████▏ | 980MB / 2.35GB  ...nts/optim_090000_rank3.pt: 90%|████████▉ | 2.23GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▌ | 19.0GB / 22.2GB, 908MB/s New Data Upload : 81%|████████ | 4.62GB / 5.71GB, 209MB/s  ...nts/optim_090000_rank1.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_090000_rank2.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_090000_rank4.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_090000_rank5.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_090000_rank6.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_090000_rank0.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_090000_rank7.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...eckpoints/model_090000.pt: 42%|████▏ | 998MB / 2.35GB  ...nts/optim_090000_rank3.pt: 90%|█████████ | 2.24GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▌ | 19.1GB / 22.2GB, 899MB/s New Data Upload : 81%|████████ | 4.65GB / 5.77GB, 209MB/s  ...nts/optim_090000_rank1.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_090000_rank2.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_090000_rank4.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_090000_rank5.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_090000_rank6.pt: 90%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_090000_rank0.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_090000_rank7.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...eckpoints/model_090000.pt: 44%|████▎ | 1.02GB / 2.35GB  ...nts/optim_090000_rank3.pt: 90%|█████████ | 2.25GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▌ | 19.2GB / 22.2GB, 890MB/s New Data Upload : 80%|████████ | 4.69GB / 5.84GB, 209MB/s  ...nts/optim_090000_rank1.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_090000_rank2.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_090000_rank4.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_090000_rank5.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_090000_rank6.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_090000_rank0.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_090000_rank7.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...eckpoints/model_090000.pt: 45%|████▍ | 1.05GB / 2.35GB  ...nts/optim_090000_rank3.pt: 91%|█████████ | 2.26GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.2GB / 22.2GB, 874MB/s New Data Upload : 80%|████████ | 4.73GB / 5.91GB, 208MB/s  ...nts/optim_090000_rank1.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_090000_rank2.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_090000_rank4.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_090000_rank5.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_090000_rank6.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_090000_rank0.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_090000_rank7.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...eckpoints/model_090000.pt: 46%|████▌ | 1.07GB / 2.35GB  ...nts/optim_090000_rank3.pt: 91%|█████████ | 2.26GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.3GB / 22.2GB, 855MB/s New Data Upload : 80%|███████▉ | 4.76GB / 5.98GB, 206MB/s  ...nts/optim_090000_rank1.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_090000_rank2.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_090000_rank4.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_090000_rank5.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_090000_rank6.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_090000_rank0.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_090000_rank7.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...eckpoints/model_090000.pt: 47%|████▋ | 1.10GB / 2.35GB  ...nts/optim_090000_rank3.pt: 91%|█████████▏| 2.27GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.4GB / 22.2GB, 849MB/s New Data Upload : 80%|████████ | 4.80GB / 5.98GB, 206MB/s  ...nts/optim_090000_rank1.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_090000_rank2.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_090000_rank4.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_090000_rank5.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_090000_rank6.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_090000_rank0.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_090000_rank7.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...eckpoints/model_090000.pt: 48%|████▊ | 1.12GB / 2.35GB  ...nts/optim_090000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.5GB / 22.2GB, 839MB/s New Data Upload : 80%|███████▉ | 4.83GB / 6.04GB, 205MB/s  ...nts/optim_090000_rank1.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_090000_rank2.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_090000_rank4.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_090000_rank5.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_090000_rank6.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_090000_rank0.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_090000_rank7.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...eckpoints/model_090000.pt: 49%|████▊ | 1.14GB / 2.35GB  ...nts/optim_090000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.5GB / 22.2GB, 827MB/s New Data Upload : 80%|███████▉ | 4.86GB / 6.11GB, 205MB/s  ...nts/optim_090000_rank1.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_090000_rank2.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_090000_rank4.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_090000_rank5.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_090000_rank6.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_090000_rank0.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_090000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...eckpoints/model_090000.pt: 50%|████▉ | 1.17GB / 2.35GB  ...nts/optim_090000_rank3.pt: 92%|█████████▏| 2.29GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.6GB / 22.2GB, 814MB/s New Data Upload : 80%|████████ | 4.90GB / 6.11GB, 204MB/s  ...nts/optim_090000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_090000_rank2.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_090000_rank4.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_090000_rank5.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_090000_rank6.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_090000_rank0.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_090000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...eckpoints/model_090000.pt: 51%|█████ | 1.20GB / 2.35GB  ...nts/optim_090000_rank3.pt: 92%|█████████▏| 2.29GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.7GB / 22.2GB, 806MB/s New Data Upload : 80%|███████▉ | 4.94GB / 6.18GB, 203MB/s  ...nts/optim_090000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_090000_rank2.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_090000_rank4.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_090000_rank5.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_090000_rank6.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_090000_rank0.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_090000_rank7.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...eckpoints/model_090000.pt: 52%|█████▏ | 1.23GB / 2.35GB  ...nts/optim_090000_rank3.pt: 92%|█████████▏| 2.30GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▊ | 19.7GB / 22.2GB, 796MB/s New Data Upload : 80%|███████▉ | 4.97GB / 6.24GB, 203MB/s  ...nts/optim_090000_rank1.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_090000_rank2.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_090000_rank4.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_090000_rank5.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_090000_rank6.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_090000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_090000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...eckpoints/model_090000.pt: 53%|█████▎ | 1.26GB / 2.35GB  ...nts/optim_090000_rank3.pt: 93%|█████████▎| 2.31GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▉ | 19.8GB / 22.2GB, 784MB/s New Data Upload : 80%|███████▉ | 5.02GB / 6.31GB, 202MB/s  ...nts/optim_090000_rank1.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_090000_rank2.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_090000_rank4.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_090000_rank5.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_090000_rank6.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_090000_rank0.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_090000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...eckpoints/model_090000.pt: 55%|█████▍ | 1.29GB / 2.35GB  ...nts/optim_090000_rank3.pt: 93%|█████████▎| 2.32GB / 2.49GB  Processing Files (0 / 9) : 90%|████████▉ | 19.9GB / 22.2GB, 777MB/s New Data Upload : 79%|███████▉ | 5.06GB / 6.38GB, 202MB/s  ...nts/optim_090000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_090000_rank2.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_090000_rank4.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_090000_rank5.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_090000_rank6.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_090000_rank0.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_090000_rank7.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...eckpoints/model_090000.pt: 56%|█████▌ | 1.31GB / 2.35GB  ...nts/optim_090000_rank3.pt: 93%|█████████▎| 2.32GB / 2.49GB  Processing Files (0 / 9) : 90%|████████▉ | 20.0GB / 22.2GB, 754MB/s New Data Upload : 79%|███████▉ | 5.10GB / 6.43GB, 201MB/s  ...nts/optim_090000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_090000_rank2.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_090000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_090000_rank5.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_090000_rank6.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_090000_rank0.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_090000_rank7.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...eckpoints/model_090000.pt: 57%|█████▋ | 1.34GB / 2.35GB  ...nts/optim_090000_rank3.pt: 94%|█████████▍| 2.33GB / 2.49GB  Processing Files (0 / 9) : 90%|█████████ | 20.1GB / 22.2GB, 735MB/s New Data Upload : 80%|███████▉ | 5.14GB / 6.43GB, 201MB/s  ...nts/optim_090000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_090000_rank2.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_090000_rank4.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_090000_rank5.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_090000_rank6.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_090000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_090000_rank7.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...eckpoints/model_090000.pt: 58%|█████▊ | 1.37GB / 2.35GB  ...nts/optim_090000_rank3.pt: 94%|█████████▍| 2.34GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.2GB / 22.2GB, 716MB/s New Data Upload : 81%|████████ | 5.18GB / 6.43GB, 202MB/s  ...nts/optim_090000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_090000_rank2.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_090000_rank4.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_090000_rank5.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_090000_rank6.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_090000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_090000_rank7.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...eckpoints/model_090000.pt: 60%|█████▉ | 1.40GB / 2.35GB  ...nts/optim_090000_rank3.pt: 94%|█████████▍| 2.35GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.3GB / 22.2GB, 692MB/s New Data Upload : 81%|████████ | 5.22GB / 6.43GB, 201MB/s  ...nts/optim_090000_rank1.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_090000_rank2.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_090000_rank4.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_090000_rank5.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_090000_rank6.pt: 94%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_090000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_090000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...eckpoints/model_090000.pt: 61%|██████ | 1.43GB / 2.35GB  ...nts/optim_090000_rank3.pt: 95%|█████████▍| 2.35GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████▏| 20.3GB / 22.2GB, 672MB/s New Data Upload : 82%|████████▏ | 5.26GB / 6.43GB, 201MB/s  ...nts/optim_090000_rank1.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_090000_rank2.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_090000_rank4.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_090000_rank5.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_090000_rank6.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_090000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_090000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...eckpoints/model_090000.pt: 62%|██████▏ | 1.46GB / 2.35GB  ...nts/optim_090000_rank3.pt: 95%|█████████▍| 2.36GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.4GB / 22.2GB, 650MB/s New Data Upload : 82%|████████▏ | 5.30GB / 6.43GB, 200MB/s  ...nts/optim_090000_rank1.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_090000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_090000_rank4.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_090000_rank5.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_090000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_090000_rank0.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_090000_rank7.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...eckpoints/model_090000.pt: 63%|██████▎ | 1.48GB / 2.35GB  ...nts/optim_090000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.5GB / 22.2GB, 630MB/s New Data Upload : 83%|████████▎ | 5.33GB / 6.43GB, 198MB/s  ...nts/optim_090000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_090000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_090000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_090000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_090000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_090000_rank0.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_090000_rank7.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...eckpoints/model_090000.pt: 64%|██████▍ | 1.51GB / 2.35GB  ...nts/optim_090000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.6GB / 22.2GB, 608MB/s New Data Upload : 84%|████████▎ | 5.37GB / 6.43GB, 197MB/s  ...nts/optim_090000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_090000_rank2.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_090000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_090000_rank5.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_090000_rank6.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_090000_rank0.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_090000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...eckpoints/model_090000.pt: 65%|██████▍ | 1.52GB / 2.35GB  ...nts/optim_090000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.6GB / 22.2GB, 593MB/s New Data Upload : 84%|████████▍ | 5.40GB / 6.43GB, 196MB/s  ...nts/optim_090000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_090000_rank2.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_090000_rank4.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_090000_rank5.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_090000_rank6.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_090000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_090000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...eckpoints/model_090000.pt: 66%|██████▌ | 1.55GB / 2.35GB  ...nts/optim_090000_rank3.pt: 96%|█████████▌| 2.39GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.7GB / 22.2GB, 570MB/s New Data Upload : 85%|████████▍ | 5.44GB / 6.43GB, 195MB/s  ...nts/optim_090000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank2.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_090000_rank4.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_090000_rank5.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_090000_rank6.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_090000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_090000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...eckpoints/model_090000.pt: 67%|██████▋ | 1.57GB / 2.35GB  ...nts/optim_090000_rank3.pt: 96%|█████████▋| 2.39GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.8GB / 22.2GB, 556MB/s New Data Upload : 85%|████████▌ | 5.47GB / 6.43GB, 194MB/s  ...nts/optim_090000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_090000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_090000_rank5.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_090000_rank6.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_090000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_090000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...eckpoints/model_090000.pt: 68%|██████▊ | 1.59GB / 2.35GB  ...nts/optim_090000_rank3.pt: 97%|█████████▋| 2.40GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▎| 20.8GB / 22.2GB, 524MB/s New Data Upload : 86%|████████▌ | 5.50GB / 6.43GB, 192MB/s  ...nts/optim_090000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_090000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_090000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_090000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_090000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_090000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...eckpoints/model_090000.pt: 69%|██████▉ | 1.62GB / 2.35GB  ...nts/optim_090000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 20.9GB / 22.2GB, 513MB/s New Data Upload : 86%|████████▌ | 5.54GB / 6.43GB, 192MB/s  ...nts/optim_090000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_090000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_090000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_090000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_090000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...eckpoints/model_090000.pt: 70%|██████▉ | 1.64GB / 2.35GB  ...nts/optim_090000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 21.0GB / 22.2GB, 500MB/s New Data Upload : 87%|████████▋ | 5.57GB / 6.43GB, 190MB/s  ...nts/optim_090000_rank1.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank2.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_090000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_090000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_090000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_090000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...eckpoints/model_090000.pt: 71%|███████ | 1.66GB / 2.35GB  ...nts/optim_090000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▍| 21.0GB / 22.2GB, 489MB/s New Data Upload : 87%|████████▋ | 5.60GB / 6.43GB, 189MB/s  ...nts/optim_090000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_090000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_090000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_090000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...eckpoints/model_090000.pt: 72%|███████▏ | 1.68GB / 2.35GB  ...nts/optim_090000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▍| 21.1GB / 22.2GB, 473MB/s New Data Upload : 88%|████████▊ | 5.64GB / 6.43GB, 187MB/s  ...nts/optim_090000_rank1.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...nts/optim_090000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_090000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank6.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_090000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...eckpoints/model_090000.pt: 73%|███████▎ | 1.71GB / 2.35GB  ...nts/optim_090000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.1GB / 22.2GB, 451MB/s New Data Upload : 88%|████████▊ | 5.67GB / 6.43GB, 186MB/s  ...nts/optim_090000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_090000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_090000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...eckpoints/model_090000.pt: 74%|███████▎ | 1.73GB / 2.35GB  ...nts/optim_090000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.2GB / 22.2GB, 439MB/s New Data Upload : 89%|████████▊ | 5.70GB / 6.43GB, 185MB/s  ...nts/optim_090000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank4.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_090000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_090000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...eckpoints/model_090000.pt: 75%|███████▍ | 1.75GB / 2.35GB  ...nts/optim_090000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.2GB / 22.2GB, 416MB/s New Data Upload : 89%|████████▉ | 5.73GB / 6.43GB, 183MB/s  ...nts/optim_090000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_090000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...eckpoints/model_090000.pt: 75%|███████▌ | 1.77GB / 2.35GB  ...nts/optim_090000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 406MB/s New Data Upload : 90%|████████▉ | 5.76GB / 6.43GB, 181MB/s  ...nts/optim_090000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_090000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank7.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_090000.pt: 76%|███████▌ | 1.79GB / 2.35GB  ...nts/optim_090000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 393MB/s New Data Upload : 90%|████████▉ | 5.78GB / 6.43GB, 179MB/s  ...nts/optim_090000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_090000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank7.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_090000.pt: 77%|███████▋ | 1.81GB / 2.35GB  ...nts/optim_090000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 385MB/s New Data Upload : 90%|█████████ | 5.81GB / 6.43GB, 178MB/s  ...nts/optim_090000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank7.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_090000.pt: 78%|███████▊ | 1.84GB / 2.35GB  ...nts/optim_090000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.4GB / 22.2GB, 376MB/s New Data Upload : 91%|█████████ | 5.85GB / 6.43GB, 177MB/s  ...nts/optim_090000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_090000.pt: 79%|███████▉ | 1.86GB / 2.35GB  ...nts/optim_090000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▋| 21.4GB / 22.2GB, 370MB/s New Data Upload : 91%|█████████ | 5.87GB / 6.43GB, 176MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_090000.pt: 80%|████████ | 1.88GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 363MB/s New Data Upload : 92%|█████████▏| 5.89GB / 6.43GB, 175MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_090000.pt: 81%|████████ | 1.90GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 354MB/s New Data Upload : 92%|█████████▏| 5.91GB / 6.43GB, 172MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_090000.pt: 82%|████████▏ | 1.92GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 351MB/s New Data Upload : 92%|█████████▏| 5.94GB / 6.43GB, 172MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_090000.pt: 82%|████████▏ | 1.94GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 343MB/s New Data Upload : 93%|█████████▎| 5.96GB / 6.43GB, 170MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank4.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_090000.pt: 83%|████████▎ | 1.95GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 340MB/s New Data Upload : 93%|█████████▎| 5.98GB / 6.43GB, 169MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank5.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...nts/optim_090000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_090000.pt: 84%|████████▎ | 1.97GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 333MB/s New Data Upload : 93%|█████████▎| 5.99GB / 6.43GB, 166MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_090000.pt: 84%|████████▍ | 1.98GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.7GB / 22.2GB, 329MB/s New Data Upload : 94%|█████████▎| 6.02GB / 6.43GB, 165MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank6.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_090000.pt: 85%|████████▍ | 2.00GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.7GB / 22.2GB, 318MB/s New Data Upload : 94%|█████████▍| 6.03GB / 6.43GB, 162MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank4.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...nts/optim_090000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_090000.pt: 86%|████████▌ | 2.02GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 317MB/s New Data Upload : 94%|█████████▍| 6.05GB / 6.43GB, 162MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_090000.pt: 87%|████████▋ | 2.03GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 309MB/s New Data Upload : 94%|█████████▍| 6.07GB / 6.43GB, 160MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_090000.pt: 87%|████████▋ | 2.04GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 301MB/s New Data Upload : 95%|█████████▍| 6.08GB / 6.43GB, 157MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_090000.pt: 88%|████████▊ | 2.06GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 297MB/s New Data Upload : 95%|█████████▍| 6.10GB / 6.43GB, 156MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_090000.pt: 88%|████████▊ | 2.07GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 290MB/s New Data Upload : 95%|█████████▌| 6.12GB / 6.43GB, 154MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_090000.pt: 89%|████████▉ | 2.09GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 287MB/s New Data Upload : 96%|█████████▌| 6.14GB / 6.43GB, 152MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_090000.pt: 90%|████████▉ | 2.11GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 281MB/s New Data Upload : 96%|█████████▌| 6.16GB / 6.43GB, 151MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_090000.pt: 91%|█████████ | 2.13GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 274MB/s New Data Upload : 96%|█████████▌| 6.18GB / 6.43GB, 150MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_090000.pt: 91%|█████████ | 2.14GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 269MB/s New Data Upload : 96%|█████████▋| 6.19GB / 6.43GB, 147MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_090000.pt: 92%|█████████▏| 2.15GB / 2.35GB  ...nts/optim_090000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 263MB/s New Data Upload : 97%|█████████▋| 6.21GB / 6.43GB, 145MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_090000.pt: 92%|█████████▏| 2.17GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 22.0GB / 22.2GB, 261MB/s New Data Upload : 97%|█████████▋| 6.23GB / 6.43GB, 144MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_090000.pt: 93%|█████████▎| 2.18GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 255MB/s New Data Upload : 97%|█████████▋| 6.25GB / 6.43GB, 142MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_090000.pt: 94%|█████████▎| 2.20GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 251MB/s New Data Upload : 97%|█████████▋| 6.27GB / 6.43GB, 141MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_090000.pt: 94%|█████████▍| 2.21GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 247MB/s New Data Upload : 98%|█████████▊| 6.28GB / 6.43GB, 139MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_090000.pt: 95%|█████████▍| 2.22GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 242MB/s New Data Upload : 98%|█████████▊| 6.29GB / 6.43GB, 137MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_090000.pt: 95%|█████████▌| 2.23GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 238MB/s New Data Upload : 98%|█████████▊| 6.31GB / 6.43GB, 135MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_090000.pt: 96%|█████████▌| 2.24GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 233MB/s New Data Upload : 98%|█████████▊| 6.32GB / 6.43GB, 132MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_090000.pt: 96%|█████████▌| 2.25GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 224MB/s New Data Upload : 98%|█████████▊| 6.33GB / 6.43GB, 129MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 96%|█████████▋| 2.26GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 218MB/s New Data Upload : 99%|█████████▊| 6.34GB / 6.43GB, 126MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 211MB/s New Data Upload : 99%|█████████▉| 6.35GB / 6.43GB, 123MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 202MB/s New Data Upload : 99%|█████████▉| 6.36GB / 6.43GB, 119MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 195MB/s New Data Upload : 99%|█████████▉| 6.36GB / 6.43GB, 116MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 97%|█████████▋| 2.29GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 188MB/s New Data Upload : 99%|█████████▉| 6.37GB / 6.43GB, 112MB/s  ...nts/optim_090000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 98%|█████████▊| 2.29GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 181MB/s New Data Upload : 99%|█████████▉| 6.37GB / 6.43GB, 109MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 175MB/s New Data Upload : 99%|█████████▉| 6.38GB / 6.43GB, 106MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 166MB/s New Data Upload : 99%|█████████▉| 6.38GB / 6.43GB, 102MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 160MB/s New Data Upload : 99%|█████████▉| 6.38GB / 6.43GB, 99.2MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 152MB/s New Data Upload : 99%|█████████▉| 6.39GB / 6.43GB, 96.4MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 146MB/s New Data Upload : 99%|█████████▉| 6.39GB / 6.43GB, 93.1MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 140MB/s New Data Upload : 99%|█████████▉| 6.40GB / 6.43GB, 90.4MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 134MB/s New Data Upload : 100%|█████████▉| 6.40GB / 6.43GB, 87.8MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 127MB/s New Data Upload : 100%|█████████▉| 6.40GB / 6.43GB, 84.3MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 120MB/s New Data Upload : 100%|█████████▉| 6.41GB / 6.43GB, 81.5MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 116MB/s New Data Upload : 100%|█████████▉| 6.41GB / 6.43GB, 78.8MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 112MB/s New Data Upload : 100%|█████████▉| 6.41GB / 6.43GB, 76.0MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 106MB/s New Data Upload : 100%|█████████▉| 6.41GB / 6.43GB, 73.1MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 101MB/s New Data Upload : 100%|█████████▉| 6.41GB / 6.43GB, 69.9MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 99%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 97.5MB/s New Data Upload : 100%|█████████▉| 6.42GB / 6.43GB, 67.4MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 93.2MB/s New Data Upload : 100%|█████████▉| 6.42GB / 6.43GB, 65.0MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 89.5MB/s New Data Upload : 100%|█████████▉| 6.42GB / 6.43GB, 62.5MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 85.9MB/s New Data Upload : 100%|█████████▉| 6.42GB / 6.43GB, 59.9MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 81.2MB/s New Data Upload : 100%|█████████▉| 6.42GB / 6.43GB, 56.6MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 78.7MB/s New Data Upload : 100%|█████████▉| 6.42GB / 6.43GB, 54.8MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 74.5MB/s New Data Upload : 100%|█████████▉| 6.43GB / 6.43GB, 52.4MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 72.5MB/s New Data Upload : 100%|█████████▉| 6.43GB / 6.43GB, 50.5MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 68.8MB/s New Data Upload : 100%|█████████▉| 6.43GB / 6.43GB, 48.3MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (9 / 9) : 100%|██████████| 22.2GB / 22.2GB, 54.7MB/s New Data Upload : 100%|██████████| 6.43GB / 6.43GB, 39.2MB/s  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_090000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (9 / 9) : 100%|██████████| 22.2GB / 22.2GB, 21.9MB/s New Data Upload : 100%|██████████| 6.43GB / 6.43GB, 16.1MB/s ...nts/optim_090000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_090000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_090000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_090000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_090000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_090000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_090000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB ...eckpoints/model_090000.pt: 100%|██████████| 2.35GB / 2.35GB ...nts/optim_090000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB [23:58:44] Step 90000 uploaded [00:58:44] Uploading step 95000... Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...nts/optim_095000_rank1.pt: 0%| | 27.8kB / 2.49GB  ...nts/optim_095000_rank3.pt: 0%| | 585kB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 27.8kB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 27.8kB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 27.8kB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 97.0kB / 2.49GB  ...nts/optim_095000_rank1.pt: 0%| | 27.8kB / 2.49GB  ...nts/optim_095000_rank3.pt: 0%| | 585kB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 27.8kB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 27.8kB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 27.8kB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 97.0kB / 2.49GB  Processing Files (0 / 6) : 0%| | 794kB / 22.2GB, 794kB/s New Data Upload : 0%| | 585kB / 604MB, 586kB/s  ...nts/optim_095000_rank1.pt: 0%| | 27.8kB / 2.49GB  ...nts/optim_095000_rank3.pt: 0%| | 585kB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 27.8kB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 27.8kB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 27.8kB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 97.0kB / 2.49GB  ...nts/optim_095000_rank1.pt: 0%| | 111kB / 2.49GB  ...nts/optim_095000_rank3.pt: 0%| | 2.34MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 111kB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 111kB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 111kB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 388kB / 2.49GB  Processing Files (0 / 6) : 0%| | 3.17MB / 22.2GB, 2.27MB/s New Data Upload : 0%| | 2.34MB / 604MB, 1.67MB/s  ...nts/optim_095000_rank1.pt: 0%| | 306kB / 2.49GB  ...nts/optim_095000_rank3.pt: 0%| | 6.44MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 306kB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 306kB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 306kB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 1.07MB / 2.49GB  Processing Files (0 / 6) : 0%| | 8.73MB / 22.2GB, 5.46MB/s New Data Upload : 1%| | 6.44MB / 604MB, 4.03MB/s  ...nts/optim_095000_rank1.pt: 0%| | 389kB / 2.49GB  ...nts/optim_095000_rank3.pt: 0%| | 8.20MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 389kB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 389kB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 389kB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 1.36MB / 2.49GB  Processing Files (0 / 6) : 0%| | 11.1MB / 22.2GB, 6.17MB/s New Data Upload : 1%|▏ | 8.20MB / 604MB, 4.56MB/s  ...nts/optim_095000_rank1.pt: 0%| | 806kB / 2.49GB  ...nts/optim_095000_rank3.pt: 1%| | 17.0MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 806kB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 806kB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 806kB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 2.81MB / 2.49GB  Processing Files (0 / 6) : 0%| | 23.0MB / 22.2GB, 11.5MB/s New Data Upload : 3%|▎ | 17.0MB / 604MB, 8.49MB/s  ...nts/optim_095000_rank1.pt: 0%| | 1.03MB / 2.49GB  ...nts/optim_095000_rank3.pt: 1%| | 21.7MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 1.03MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 1.03MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 1.03MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 3.59MB / 2.49GB  Processing Files (0 / 6) : 0%| | 29.4MB / 22.2GB, 13.4MB/s New Data Upload : 4%|▎ | 21.7MB / 604MB, 9.85MB/s  ...nts/optim_095000_rank1.pt: 0%| | 1.42MB / 2.49GB  ...nts/optim_095000_rank3.pt: 1%| | 29.9MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 1.42MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 1.42MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 1.42MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 4.94MB / 2.49GB  Processing Files (0 / 6) : 0%| | 40.5MB / 22.2GB, 16.9MB/s New Data Upload : 5%|▍ | 29.9MB / 604MB, 12.4MB/s  ...nts/optim_095000_rank1.pt: 0%| | 1.64MB / 2.49GB  ...nts/optim_095000_rank3.pt: 1%|▏ | 34.5MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 1.64MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 1.64MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 1.64MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 5.72MB / 2.49GB  Processing Files (0 / 6) : 0%| | 46.8MB / 22.2GB, 18.0MB/s New Data Upload : 6%|▌ | 34.5MB / 604MB, 13.3MB/s  ...nts/optim_095000_rank1.pt: 0%| | 2.06MB / 2.49GB  ...nts/optim_095000_rank3.pt: 2%|▏ | 43.3MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 2.06MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 2.06MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 2.06MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 7.17MB / 2.49GB  Processing Files (0 / 6) : 0%| | 58.7MB / 22.2GB, 21.0MB/s New Data Upload : 7%|▋ | 43.3MB / 604MB, 15.5MB/s  ...nts/optim_095000_rank1.pt: 0%| | 2.39MB / 2.49GB  ...nts/optim_095000_rank3.pt: 2%|▏ | 50.3MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 2.39MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 2.39MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 2.39MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 8.34MB / 2.49GB  Processing Files (0 / 6) : 0%| | 68.2MB / 22.2GB, 22.8MB/s New Data Upload : 8%|▊ | 50.3MB / 604MB, 16.8MB/s  ...nts/optim_095000_rank1.pt: 0%| | 2.73MB / 2.49GB  ...nts/optim_095000_rank3.pt: 2%|▏ | 57.4MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 2.73MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 2.73MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 2.73MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 9.50MB / 2.49GB  Processing Files (0 / 6) : 0%| | 77.8MB / 22.2GB, 24.3MB/s New Data Upload : 10%|▉ | 57.4MB / 604MB, 17.9MB/s  ...nts/optim_095000_rank1.pt: 0%| | 2.92MB / 2.49GB  ...nts/optim_095000_rank3.pt: 2%|▏ | 61.5MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 2.92MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 2.92MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 2.92MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 10.2MB / 2.49GB  Processing Files (0 / 6) : 0%| | 83.3MB / 22.2GB, 24.5MB/s New Data Upload : 10%|█ | 61.5MB / 604MB, 18.1MB/s  ...nts/optim_095000_rank1.pt: 0%| | 3.17MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 66.7MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 3.17MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 3.17MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 3.17MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 11.1MB / 2.49GB  Processing Files (0 / 6) : 0%| | 90.5MB / 22.2GB, 25.1MB/s New Data Upload : 11%|█ | 66.7MB / 604MB, 18.5MB/s  ...nts/optim_095000_rank1.pt: 0%| | 3.17MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 66.7MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 3.17MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 3.17MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 3.17MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_095000_rank1.pt: 0%| | 3.17MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 66.7MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 3.17MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 3.17MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 3.17MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_095000_rank1.pt: 0%| | 3.17MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 66.7MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 3.17MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 3.17MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 3.17MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_095000_rank0.pt: 0%| | 3.19MB / 2.49GB  ...nts/optim_095000_rank1.pt: 0%| | 3.19MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 3.19MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 3.19MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 3.19MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 11.1MB / 2.49GB  ...nts/optim_095000_rank0.pt: 0%| | 3.19MB / 2.49GB  Processing Files (0 / 7) : 0%| | 94.1MB / 22.2GB, 21.4MB/s New Data Upload : 11%|█ | 67.1MB / 604MB, 15.3MB/s  ...nts/optim_095000_rank1.pt: 0%| | 3.30MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 3.88MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 3.31MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 3.30MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 11.2MB / 2.49GB  ...nts/optim_095000_rank0.pt: 0%| | 5.54MB / 2.49GB  Processing Files (0 / 7) : 0%| | 97.7MB / 22.2GB, 21.2MB/s New Data Upload : 9%|▉ | 70.0MB / 738MB, 15.2MB/s  ...nts/optim_095000_rank1.pt: 0%| | 3.30MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 3.88MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 3.31MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 3.30MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 11.2MB / 2.49GB  ...nts/optim_095000_rank0.pt: 0%| | 5.54MB / 2.49GB  ...nts/optim_095000_rank1.pt: 0%| | 3.34MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 67.1MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 5.05MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 3.34MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 3.30MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 11.3MB / 2.49GB  ...nts/optim_095000_rank0.pt: 0%| | 5.57MB / 2.49GB  Processing Files (0 / 7) : 0%| | 99.0MB / 22.2GB, 19.8MB/s New Data Upload : 10%|▉ | 71.2MB / 738MB, 14.2MB/s  ...nts/optim_095000_rank1.pt: 0%| | 4.19MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 67.2MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 8.76MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 3.62MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 3.51MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 11.5MB / 2.49GB  ...nts/optim_095000_rank0.pt: 0%| | 9.74MB / 2.49GB  Processing Files (0 / 7) : 0%| | 109MB / 22.2GB, 20.9MB/s New Data Upload : 11%|█ | 79.4MB / 738MB, 15.3MB/s  ...nts/optim_095000_rank1.pt: 0%| | 7.25MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 67.3MB / 2.49GB  ...nts/optim_095000_rank4.pt: 0%| | 11.4MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 3.93MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 3.76MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 11.9MB / 2.49GB  ...nts/optim_095000_rank0.pt: 1%| | 15.1MB / 2.49GB  Processing Files (0 / 7) : 1%| | 120MB / 22.2GB, 22.3MB/s New Data Upload : 11%|█ | 87.0MB / 805MB, 16.1MB/s  ...nts/optim_095000_rank1.pt: 0%| | 8.74MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 67.6MB / 2.49GB  ...nts/optim_095000_rank4.pt: 1%| | 17.4MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 8.67MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 3.98MB / 2.49GB  ...nts/optim_095000_rank7.pt: 0%| | 12.4MB / 2.49GB  ...nts/optim_095000_rank0.pt: 1%| | 19.5MB / 2.49GB  Processing Files (0 / 7) : 1%| | 138MB / 22.2GB, 24.7MB/s New Data Upload : 13%|█▎ | 102MB / 805MB, 18.3MB/s  ...nts/optim_095000_rank1.pt: 1%| | 14.0MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 67.7MB / 2.49GB  ...nts/optim_095000_rank4.pt: 1%| | 24.8MB / 2.49GB  ...nts/optim_095000_rank5.pt: 0%| | 12.2MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 4.52MB / 2.49GB  ...nts/optim_095000_rank7.pt: 1%| | 13.1MB / 2.49GB  ...nts/optim_095000_rank0.pt: 1%| | 28.4MB / 2.49GB  Processing Files (0 / 7) : 1%| | 165MB / 22.2GB, 28.4MB/s New Data Upload : 15%|█▌ | 123MB / 805MB, 21.2MB/s  ...nts/optim_095000_rank1.pt: 1%| | 16.4MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 68.0MB / 2.49GB  ...nts/optim_095000_rank4.pt: 1%|▏ | 34.1MB / 2.49GB  ...nts/optim_095000_rank5.pt: 1%| | 15.4MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 5.96MB / 2.49GB  ...nts/optim_095000_rank7.pt: 1%| | 13.9MB / 2.49GB  ...nts/optim_095000_rank0.pt: 2%|▏ | 38.1MB / 2.49GB  Processing Files (0 / 7) : 1%| | 192MB / 22.2GB, 32.0MB/s New Data Upload : 17%|█▋ | 145MB / 872MB, 24.2MB/s  ...nts/optim_095000_rank1.pt: 1%| | 21.6MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 68.3MB / 2.49GB  ...nts/optim_095000_rank4.pt: 2%|▏ | 39.2MB / 2.49GB  ...nts/optim_095000_rank5.pt: 1%| | 17.2MB / 2.49GB  ...nts/optim_095000_rank6.pt: 0%| | 11.7MB / 2.49GB  ...nts/optim_095000_rank7.pt: 1%| | 14.7MB / 2.49GB  ...nts/optim_095000_rank0.pt: 2%|▏ | 47.1MB / 2.49GB  Processing Files (0 / 7) : 1%| | 220MB / 22.2GB, 35.5MB/s New Data Upload : 19%|█▉ | 167MB / 872MB, 26.9MB/s  ...nts/optim_095000_rank1.pt: 1%| | 26.3MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 68.5MB / 2.49GB  ...nts/optim_095000_rank4.pt: 2%|▏ | 47.2MB / 2.49GB  ...nts/optim_095000_rank5.pt: 1%| | 19.1MB / 2.49GB  ...nts/optim_095000_rank6.pt: 1%| | 12.5MB / 2.49GB  ...nts/optim_095000_rank7.pt: 1%| | 15.4MB / 2.49GB  ...nts/optim_095000_rank0.pt: 2%|▏ | 55.6MB / 2.49GB  Processing Files (0 / 7) : 1%| | 244MB / 22.2GB, 38.2MB/s New Data Upload : 22%|██▏ | 188MB / 872MB, 29.3MB/s  ...nts/optim_095000_rank1.pt: 1%| | 26.9MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 68.9MB / 2.49GB  ...nts/optim_095000_rank4.pt: 2%|▏ | 56.4MB / 2.49GB  ...nts/optim_095000_rank5.pt: 1%| | 23.4MB / 2.49GB  ...nts/optim_095000_rank6.pt: 1%| | 13.2MB / 2.49GB  ...nts/optim_095000_rank7.pt: 1%| | 16.2MB / 2.49GB  ...nts/optim_095000_rank0.pt: 3%|▎ | 64.7MB / 2.49GB  Processing Files (0 / 7) : 1%| | 270MB / 22.2GB, 40.9MB/s New Data Upload : 22%|██▏ | 209MB / 939MB, 31.6MB/s  ...eckpoints/model_095000.pt: 0%| | 3.66MB / 2.35GB  ...nts/optim_095000_rank1.pt: 1%| | 29.0MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 69.3MB / 2.49GB  ...nts/optim_095000_rank4.pt: 3%|▎ | 66.0MB / 2.49GB  ...nts/optim_095000_rank5.pt: 1%| | 25.9MB / 2.49GB  ...nts/optim_095000_rank6.pt: 1%| | 15.5MB / 2.49GB  ...nts/optim_095000_rank7.pt: 1%| | 16.8MB / 2.49GB  ...nts/optim_095000_rank0.pt: 3%|▎ | 69.8MB / 2.49GB  ...eckpoints/model_095000.pt: 0%| | 3.66MB / 2.35GB  Processing Files (0 / 8) : 1%|▏ | 296MB / 22.2GB, 43.5MB/s New Data Upload : 23%|██▎ | 229MB / 1.01GB, 33.6MB/s  ...nts/optim_095000_rank1.pt: 1%|▏ | 33.9MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 69.7MB / 2.49GB  ...nts/optim_095000_rank4.pt: 3%|▎ | 70.2MB / 2.49GB  ...nts/optim_095000_rank5.pt: 1%| | 29.7MB / 2.49GB  ...nts/optim_095000_rank6.pt: 1%| | 20.1MB / 2.49GB  ...nts/optim_095000_rank7.pt: 1%| | 17.5MB / 2.49GB  ...nts/optim_095000_rank0.pt: 3%|▎ | 72.6MB / 2.49GB  ...eckpoints/model_095000.pt: 0%| | 3.66MB / 2.35GB  Processing Files (0 / 8) : 1%|▏ | 317MB / 22.2GB, 45.3MB/s New Data Upload : 25%|██▍ | 247MB / 1.01GB, 35.3MB/s  ...nts/optim_095000_rank1.pt: 1%|▏ | 35.8MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 69.8MB / 2.49GB  ...nts/optim_095000_rank4.pt: 3%|▎ | 73.1MB / 2.49GB  ...nts/optim_095000_rank5.pt: 1%| | 29.8MB / 2.49GB  ...nts/optim_095000_rank6.pt: 1%| | 20.2MB / 2.49GB  ...nts/optim_095000_rank7.pt: 1%| | 17.6MB / 2.49GB  ...nts/optim_095000_rank0.pt: 3%|▎ | 72.6MB / 2.49GB  ...eckpoints/model_095000.pt: 0%| | 5.39MB / 2.35GB  Processing Files (0 / 8) : 1%|▏ | 324MB / 22.2GB, 45.0MB/s New Data Upload : 25%|██▌ | 254MB / 1.01GB, 35.3MB/s  ...nts/optim_095000_rank1.pt: 2%|▏ | 37.5MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 70.2MB / 2.49GB  ...nts/optim_095000_rank4.pt: 3%|▎ | 73.1MB / 2.49GB  ...nts/optim_095000_rank5.pt: 1%|▏ | 33.3MB / 2.49GB  ...nts/optim_095000_rank6.pt: 1%| | 25.1MB / 2.49GB  ...nts/optim_095000_rank7.pt: 1%| | 20.8MB / 2.49GB  ...nts/optim_095000_rank0.pt: 3%|▎ | 73.0MB / 2.49GB  ...eckpoints/model_095000.pt: 0%| | 9.44MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 342MB / 22.2GB, 46.3MB/s New Data Upload : 27%|██▋ | 268MB / 1.01GB, 36.2MB/s  ...nts/optim_095000_rank1.pt: 2%|▏ | 41.7MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 70.5MB / 2.49GB  ...nts/optim_095000_rank4.pt: 3%|▎ | 73.1MB / 2.49GB  ...nts/optim_095000_rank5.pt: 1%|▏ | 35.7MB / 2.49GB  ...nts/optim_095000_rank6.pt: 1%| | 28.3MB / 2.49GB  ...nts/optim_095000_rank7.pt: 1%| | 22.3MB / 2.49GB  ...nts/optim_095000_rank0.pt: 3%|▎ | 73.3MB / 2.49GB  ...eckpoints/model_095000.pt: 0%| | 11.2MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 356MB / 22.2GB, 46.9MB/s New Data Upload : 28%|██▊ | 280MB / 1.01GB, 36.9MB/s  ...nts/optim_095000_rank1.pt: 2%|▏ | 47.9MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 76.3MB / 2.49GB  ...nts/optim_095000_rank4.pt: 3%|▎ | 75.9MB / 2.49GB  ...nts/optim_095000_rank5.pt: 2%|▏ | 39.7MB / 2.49GB  ...nts/optim_095000_rank6.pt: 1%|▏ | 35.9MB / 2.49GB  ...nts/optim_095000_rank7.pt: 1%| | 29.2MB / 2.49GB  ...nts/optim_095000_rank0.pt: 3%|▎ | 82.6MB / 2.49GB  ...eckpoints/model_095000.pt: 1%| | 14.1MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 402MB / 22.2GB, 51.5MB/s New Data Upload : 28%|██▊ | 303MB / 1.07GB, 38.8MB/s  ...nts/optim_095000_rank1.pt: 2%|▏ | 56.7MB / 2.49GB  ...nts/optim_095000_rank3.pt: 3%|▎ | 83.0MB / 2.49GB  ...nts/optim_095000_rank4.pt: 3%|▎ | 86.4MB / 2.49GB  ...nts/optim_095000_rank5.pt: 2%|▏ | 49.0MB / 2.49GB  ...nts/optim_095000_rank6.pt: 2%|▏ | 46.0MB / 2.49GB  ...nts/optim_095000_rank7.pt: 2%|▏ | 39.3MB / 2.49GB  ...nts/optim_095000_rank0.pt: 4%|▎ | 90.0MB / 2.49GB  ...eckpoints/model_095000.pt: 1%| | 14.1MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 464MB / 22.2GB, 58.0MB/s New Data Upload : 30%|███ | 324MB / 1.07GB, 40.5MB/s  ...nts/optim_095000_rank1.pt: 3%|▎ | 63.1MB / 2.49GB  ...nts/optim_095000_rank3.pt: 4%|▎ | 88.1MB / 2.49GB  ...nts/optim_095000_rank4.pt: 4%|▎ | 91.1MB / 2.49GB  ...nts/optim_095000_rank5.pt: 2%|▏ | 55.4MB / 2.49GB  ...nts/optim_095000_rank6.pt: 2%|▏ | 54.3MB / 2.49GB  ...nts/optim_095000_rank7.pt: 2%|▏ | 46.6MB / 2.49GB  ...nts/optim_095000_rank0.pt: 4%|▍ | 98.5MB / 2.49GB  ...eckpoints/model_095000.pt: 1%| | 19.3MB / 2.35GB  Processing Files (0 / 8) : 2%|▏ | 517MB / 22.2GB, 63.0MB/s New Data Upload : 32%|███▏ | 347MB / 1.07GB, 42.3MB/s  ...nts/optim_095000_rank1.pt: 3%|▎ | 73.0MB / 2.49GB  ...nts/optim_095000_rank3.pt: 4%|▍ | 96.5MB / 2.49GB  ...nts/optim_095000_rank4.pt: 4%|▍ | 99.5MB / 2.49GB  ...nts/optim_095000_rank5.pt: 3%|▎ | 65.3MB / 2.49GB  ...nts/optim_095000_rank6.pt: 3%|▎ | 64.3MB / 2.49GB  ...nts/optim_095000_rank7.pt: 2%|▏ | 58.7MB / 2.49GB  ...nts/optim_095000_rank0.pt: 4%|▍ | 109MB / 2.49GB  ...eckpoints/model_095000.pt: 1%| | 23.9MB / 2.35GB  Processing Files (0 / 8) : 3%|▎ | 591MB / 22.2GB, 70.3MB/s New Data Upload : 35%|███▍ | 374MB / 1.07GB, 44.6MB/s  ...nts/optim_095000_rank1.pt: 3%|▎ | 79.7MB / 2.49GB  ...nts/optim_095000_rank3.pt: 4%|▍ | 102MB / 2.49GB  ...nts/optim_095000_rank4.pt: 4%|▍ | 105MB / 2.49GB  ...nts/optim_095000_rank5.pt: 3%|▎ | 72.6MB / 2.49GB  ...nts/optim_095000_rank6.pt: 3%|▎ | 72.4MB / 2.49GB  ...nts/optim_095000_rank7.pt: 3%|▎ | 66.4MB / 2.49GB  ...nts/optim_095000_rank0.pt: 5%|▍ | 115MB / 2.49GB  ...eckpoints/model_095000.pt: 1%| | 28.0MB / 2.35GB  Processing Files (0 / 8) : 3%|▎ | 640MB / 22.2GB, 74.5MB/s New Data Upload : 35%|███▍ | 395MB / 1.14GB, 46.0MB/s  ...nts/optim_095000_rank1.pt: 4%|▎ | 90.3MB / 2.49GB  ...nts/optim_095000_rank3.pt: 4%|▍ | 111MB / 2.49GB  ...nts/optim_095000_rank4.pt: 5%|▍ | 113MB / 2.49GB  ...nts/optim_095000_rank5.pt: 3%|▎ | 83.1MB / 2.49GB  ...nts/optim_095000_rank6.pt: 3%|▎ | 83.6MB / 2.49GB  ...nts/optim_095000_rank7.pt: 3%|▎ | 77.9MB / 2.49GB  ...nts/optim_095000_rank0.pt: 5%|▌ | 126MB / 2.49GB  ...eckpoints/model_095000.pt: 1%|▏ | 34.3MB / 2.35GB  Processing Files (0 / 8) : 3%|▎ | 720MB / 22.2GB, 81.8MB/s New Data Upload : 35%|███▌ | 423MB / 1.21GB, 48.0MB/s  ...nts/optim_095000_rank1.pt: 4%|▍ | 103MB / 2.49GB  ...nts/optim_095000_rank3.pt: 5%|▍ | 122MB / 2.49GB  ...nts/optim_095000_rank4.pt: 5%|▌ | 125MB / 2.49GB  ...nts/optim_095000_rank5.pt: 4%|▍ | 95.8MB / 2.49GB  ...nts/optim_095000_rank6.pt: 4%|▍ | 98.7MB / 2.49GB  ...nts/optim_095000_rank7.pt: 4%|▍ | 96.0MB / 2.49GB  ...nts/optim_095000_rank0.pt: 6%|▌ | 139MB / 2.49GB  ...eckpoints/model_095000.pt: 2%|▏ | 36.6MB / 2.35GB  Processing Files (0 / 8) : 4%|▎ | 816MB / 22.2GB, 90.6MB/s New Data Upload : 36%|███▌ | 456MB / 1.27GB, 50.6MB/s  ...nts/optim_095000_rank1.pt: 5%|▍ | 114MB / 2.49GB  ...nts/optim_095000_rank3.pt: 5%|▌ | 132MB / 2.49GB  ...nts/optim_095000_rank4.pt: 5%|▌ | 133MB / 2.49GB  ...nts/optim_095000_rank5.pt: 4%|▍ | 106MB / 2.49GB  ...nts/optim_095000_rank6.pt: 4%|▍ | 108MB / 2.49GB  ...nts/optim_095000_rank7.pt: 4%|▍ | 111MB / 2.49GB  ...nts/optim_095000_rank0.pt: 6%|▌ | 150MB / 2.49GB  ...eckpoints/model_095000.pt: 2%|▏ | 38.9MB / 2.35GB  Processing Files (0 / 8) : 4%|▍ | 894MB / 22.2GB, 97.1MB/s New Data Upload : 34%|███▍ | 481MB / 1.41GB, 52.3MB/s  ...nts/optim_095000_rank1.pt: 5%|▌ | 135MB / 2.49GB  ...nts/optim_095000_rank3.pt: 6%|▌ | 153MB / 2.49GB  ...nts/optim_095000_rank4.pt: 6%|▌ | 154MB / 2.49GB  ...nts/optim_095000_rank5.pt: 5%|▌ | 127MB / 2.49GB  ...nts/optim_095000_rank6.pt: 5%|▌ | 133MB / 2.49GB  ...nts/optim_095000_rank7.pt: 5%|▌ | 135MB / 2.49GB  ...nts/optim_095000_rank0.pt: 7%|▋ | 173MB / 2.49GB  ...eckpoints/model_095000.pt: 2%|▏ | 45.3MB / 2.35GB  Processing Files (0 / 8) : 5%|▍ | 1.06GB / 22.2GB, 112MB/s New Data Upload : 37%|███▋ | 521MB / 1.41GB, 55.4MB/s  ...nts/optim_095000_rank1.pt: 6%|▌ | 151MB / 2.49GB  ...nts/optim_095000_rank3.pt: 7%|▋ | 169MB / 2.49GB  ...nts/optim_095000_rank4.pt: 7%|▋ | 170MB / 2.49GB  ...nts/optim_095000_rank5.pt: 6%|▌ | 146MB / 2.49GB  ...nts/optim_095000_rank6.pt: 6%|▌ | 150MB / 2.49GB  ...nts/optim_095000_rank7.pt: 6%|▋ | 156MB / 2.49GB  ...nts/optim_095000_rank0.pt: 8%|▊ | 190MB / 2.49GB  ...eckpoints/model_095000.pt: 2%|▏ | 48.8MB / 2.35GB  Processing Files (0 / 8) : 5%|▌ | 1.18GB / 22.2GB, 123MB/s New Data Upload : 40%|███▉ | 556MB / 1.41GB, 58.0MB/s  ...nts/optim_095000_rank1.pt: 7%|▋ | 174MB / 2.49GB  ...nts/optim_095000_rank3.pt: 8%|▊ | 191MB / 2.49GB  ...nts/optim_095000_rank4.pt: 8%|▊ | 191MB / 2.49GB  ...nts/optim_095000_rank5.pt: 7%|▋ | 168MB / 2.49GB  ...nts/optim_095000_rank6.pt: 7%|▋ | 173MB / 2.49GB  ...nts/optim_095000_rank7.pt: 7%|▋ | 180MB / 2.49GB  ...nts/optim_095000_rank0.pt: 9%|▊ | 213MB / 2.49GB  ...eckpoints/model_095000.pt: 2%|▏ | 52.8MB / 2.35GB  Processing Files (0 / 8) : 6%|▌ | 1.35GB / 22.2GB, 137MB/s New Data Upload : 40%|████ | 593MB / 1.48GB, 60.6MB/s  ...nts/optim_095000_rank1.pt: 8%|▊ | 206MB / 2.49GB  ...nts/optim_095000_rank3.pt: 9%|▉ | 223MB / 2.49GB  ...nts/optim_095000_rank4.pt: 9%|▉ | 222MB / 2.49GB  ...nts/optim_095000_rank5.pt: 8%|▊ | 201MB / 2.49GB  ...nts/optim_095000_rank6.pt: 8%|▊ | 208MB / 2.49GB  ...nts/optim_095000_rank7.pt: 9%|▊ | 214MB / 2.49GB  ...nts/optim_095000_rank0.pt: 10%|▉ | 246MB / 2.49GB  ...eckpoints/model_095000.pt: 2%|▏ | 55.1MB / 2.35GB  Processing Files (0 / 8) : 7%|▋ | 1.57GB / 22.2GB, 157MB/s New Data Upload : 42%|████▏ | 643MB / 1.54GB, 64.3MB/s  ...nts/optim_095000_rank1.pt: 9%|▉ | 221MB / 2.49GB  ...nts/optim_095000_rank3.pt: 10%|▉ | 236MB / 2.49GB  ...nts/optim_095000_rank4.pt: 9%|▉ | 236MB / 2.49GB  ...nts/optim_095000_rank5.pt: 9%|▊ | 214MB / 2.49GB  ...nts/optim_095000_rank6.pt: 9%|▉ | 223MB / 2.49GB  ...nts/optim_095000_rank7.pt: 9%|▉ | 229MB / 2.49GB  ...nts/optim_095000_rank0.pt: 10%|█ | 261MB / 2.49GB  ...eckpoints/model_095000.pt: 2%|▏ | 55.7MB / 2.35GB  Processing Files (0 / 8) : 8%|▊ | 1.68GB / 22.2GB, 164MB/s New Data Upload : 43%|████▎ | 663MB / 1.54GB, 65.0MB/s  ...nts/optim_095000_rank1.pt: 10%|▉ | 241MB / 2.49GB  ...nts/optim_095000_rank3.pt: 10%|█ | 255MB / 2.49GB  ...nts/optim_095000_rank4.pt: 10%|█ | 255MB / 2.49GB  ...nts/optim_095000_rank5.pt: 9%|▉ | 236MB / 2.49GB  ...nts/optim_095000_rank6.pt: 10%|▉ | 242MB / 2.49GB  ...nts/optim_095000_rank7.pt: 10%|█ | 254MB / 2.49GB  ...nts/optim_095000_rank0.pt: 11%|█▏ | 281MB / 2.49GB  ...eckpoints/model_095000.pt: 3%|▎ | 59.7MB / 2.35GB  Processing Files (0 / 8) : 8%|▊ | 1.82GB / 22.2GB, 179MB/s New Data Upload : 45%|████▌ | 701MB / 1.54GB, 68.7MB/s  ...nts/optim_095000_rank1.pt: 11%|█ | 265MB / 2.49GB  ...nts/optim_095000_rank3.pt: 11%|█ | 277MB / 2.49GB  ...nts/optim_095000_rank4.pt: 11%|█ | 276MB / 2.49GB  ...nts/optim_095000_rank5.pt: 10%|█ | 258MB / 2.49GB  ...nts/optim_095000_rank6.pt: 11%|█ | 268MB / 2.49GB  ...nts/optim_095000_rank7.pt: 11%|█ | 274MB / 2.49GB  ...nts/optim_095000_rank0.pt: 12%|█▏ | 304MB / 2.49GB  ...eckpoints/model_095000.pt: 3%|▎ | 67.7MB / 2.35GB  Processing Files (0 / 8) : 9%|▉ | 1.99GB / 22.2GB, 195MB/s New Data Upload : 46%|████▌ | 739MB / 1.61GB, 72.5MB/s  ...nts/optim_095000_rank1.pt: 11%|█ | 275MB / 2.49GB  ...nts/optim_095000_rank3.pt: 12%|█▏ | 286MB / 2.49GB  ...nts/optim_095000_rank4.pt: 11%|█▏ | 285MB / 2.49GB  ...nts/optim_095000_rank5.pt: 11%|█ | 269MB / 2.49GB  ...nts/optim_095000_rank6.pt: 11%|█ | 279MB / 2.49GB  ...nts/optim_095000_rank7.pt: 11%|█▏ | 283MB / 2.49GB  ...nts/optim_095000_rank0.pt: 13%|█▎ | 314MB / 2.49GB  ...eckpoints/model_095000.pt: 3%|▎ | 75.0MB / 2.35GB  Processing Files (0 / 8) : 9%|▉ | 2.07GB / 22.2GB, 203MB/s New Data Upload : 47%|████▋ | 762MB / 1.61GB, 74.7MB/s  ...nts/optim_095000_rank1.pt: 12%|█▏ | 293MB / 2.49GB  ...nts/optim_095000_rank3.pt: 12%|█▏ | 302MB / 2.49GB  ...nts/optim_095000_rank4.pt: 12%|█▏ | 301MB / 2.49GB  ...nts/optim_095000_rank5.pt: 11%|█▏ | 285MB / 2.49GB  ...nts/optim_095000_rank6.pt: 12%|█▏ | 296MB / 2.49GB  ...nts/optim_095000_rank7.pt: 12%|█▏ | 299MB / 2.49GB  ...nts/optim_095000_rank0.pt: 13%|█▎ | 329MB / 2.49GB  ...eckpoints/model_095000.pt: 3%|▎ | 79.6MB / 2.35GB  Processing Files (0 / 8) : 10%|▉ | 2.18GB / 22.2GB, 214MB/s New Data Upload : 49%|████▉ | 787MB / 1.61GB, 77.2MB/s  ...nts/optim_095000_rank1.pt: 12%|█▏ | 310MB / 2.49GB  ...nts/optim_095000_rank3.pt: 13%|█▎ | 319MB / 2.49GB  ...nts/optim_095000_rank4.pt: 13%|█▎ | 318MB / 2.49GB  ...nts/optim_095000_rank5.pt: 12%|█▏ | 303MB / 2.49GB  ...nts/optim_095000_rank6.pt: 13%|█▎ | 314MB / 2.49GB  ...nts/optim_095000_rank7.pt: 13%|█▎ | 316MB / 2.49GB  ...nts/optim_095000_rank0.pt: 14%|█▍ | 346MB / 2.49GB  ...eckpoints/model_095000.pt: 4%|▎ | 85.8MB / 2.35GB  Processing Files (0 / 8) : 10%|█ | 2.31GB / 22.2GB, 227MB/s New Data Upload : 51%|█████ | 816MB / 1.61GB, 79.9MB/s  ...nts/optim_095000_rank1.pt: 13%|█▎ | 329MB / 2.49GB  ...nts/optim_095000_rank3.pt: 13%|█▎ | 332MB / 2.49GB  ...nts/optim_095000_rank4.pt: 13%|█▎ | 332MB / 2.49GB  ...nts/optim_095000_rank5.pt: 13%|█▎ | 316MB / 2.49GB  ...nts/optim_095000_rank6.pt: 13%|█▎ | 329MB / 2.49GB  ...nts/optim_095000_rank7.pt: 13%|█▎ | 330MB / 2.49GB  ...nts/optim_095000_rank0.pt: 14%|█▍ | 359MB / 2.49GB  ...eckpoints/model_095000.pt: 4%|▍ | 89.7MB / 2.35GB  Processing Files (0 / 8) : 11%|█ | 2.42GB / 22.2GB, 237MB/s New Data Upload : 52%|█████▏ | 841MB / 1.61GB, 82.4MB/s  ...nts/optim_095000_rank1.pt: 14%|█▍ | 344MB / 2.49GB  ...nts/optim_095000_rank3.pt: 14%|█▍ | 346MB / 2.49GB  ...nts/optim_095000_rank4.pt: 14%|█▍ | 351MB / 2.49GB  ...nts/optim_095000_rank5.pt: 13%|█▎ | 332MB / 2.49GB  ...nts/optim_095000_rank6.pt: 14%|█▍ | 345MB / 2.49GB  ...nts/optim_095000_rank7.pt: 14%|█▍ | 345MB / 2.49GB  ...nts/optim_095000_rank0.pt: 15%|█▌ | 375MB / 2.49GB  ...eckpoints/model_095000.pt: 4%|▍ | 91.4MB / 2.35GB  Processing Files (0 / 8) : 11%|█▏ | 2.53GB / 22.2GB, 248MB/s New Data Upload : 52%|█████▏ | 866MB / 1.68GB, 84.7MB/s  ...nts/optim_095000_rank1.pt: 14%|█▍ | 356MB / 2.49GB  ...nts/optim_095000_rank3.pt: 14%|█▍ | 356MB / 2.49GB  ...nts/optim_095000_rank4.pt: 15%|█▍ | 361MB / 2.49GB  ...nts/optim_095000_rank5.pt: 14%|█▍ | 342MB / 2.49GB  ...nts/optim_095000_rank6.pt: 14%|█▍ | 355MB / 2.49GB  ...nts/optim_095000_rank7.pt: 14%|█▍ | 354MB / 2.49GB  ...nts/optim_095000_rank0.pt: 16%|█▌ | 386MB / 2.49GB  ...eckpoints/model_095000.pt: 4%|▍ | 96.5MB / 2.35GB  Processing Files (0 / 8) : 12%|█▏ | 2.61GB / 22.2GB, 255MB/s New Data Upload : 51%|█████ | 886MB / 1.74GB, 86.2MB/s  ...nts/optim_095000_rank1.pt: 15%|█▍ | 370MB / 2.49GB  ...nts/optim_095000_rank3.pt: 15%|█▍ | 364MB / 2.49GB  ...nts/optim_095000_rank4.pt: 15%|█▌ | 373MB / 2.49GB  ...nts/optim_095000_rank5.pt: 14%|█▍ | 352MB / 2.49GB  ...nts/optim_095000_rank6.pt: 15%|█▍ | 365MB / 2.49GB  ...nts/optim_095000_rank7.pt: 15%|█▍ | 363MB / 2.49GB  ...nts/optim_095000_rank0.pt: 16%|█▌ | 397MB / 2.49GB  ...eckpoints/model_095000.pt: 4%|▍ | 102MB / 2.35GB  Processing Files (0 / 8) : 12%|█▏ | 2.69GB / 22.2GB, 262MB/s New Data Upload : 52%|█████▏ | 914MB / 1.74GB, 88.8MB/s  ...nts/optim_095000_rank1.pt: 15%|█▌ | 383MB / 2.49GB  ...nts/optim_095000_rank3.pt: 15%|█▌ | 375MB / 2.49GB  ...nts/optim_095000_rank4.pt: 16%|█▌ | 387MB / 2.49GB  ...nts/optim_095000_rank5.pt: 15%|█▍ | 363MB / 2.49GB  ...nts/optim_095000_rank6.pt: 15%|█▌ | 376MB / 2.49GB  ...nts/optim_095000_rank7.pt: 15%|█▌ | 375MB / 2.49GB  ...nts/optim_095000_rank0.pt: 16%|█▋ | 409MB / 2.49GB  ...eckpoints/model_095000.pt: 5%|▍ | 108MB / 2.35GB  Processing Files (0 / 8) : 12%|█▏ | 2.78GB / 22.2GB, 270MB/s New Data Upload : 52%|█████▏ | 936MB / 1.81GB, 90.1MB/s  ...nts/optim_095000_rank1.pt: 16%|█▌ | 395MB / 2.49GB  ...nts/optim_095000_rank3.pt: 16%|█▌ | 386MB / 2.49GB  ...nts/optim_095000_rank4.pt: 16%|█▌ | 396MB / 2.49GB  ...nts/optim_095000_rank5.pt: 15%|█▌ | 375MB / 2.49GB  ...nts/optim_095000_rank6.pt: 16%|█▌ | 386MB / 2.49GB  ...nts/optim_095000_rank7.pt: 16%|█▌ | 385MB / 2.49GB  ...nts/optim_095000_rank0.pt: 17%|█▋ | 417MB / 2.49GB  ...eckpoints/model_095000.pt: 5%|▍ | 110MB / 2.35GB  Processing Files (0 / 8) : 13%|█▎ | 2.85GB / 22.2GB, 277MB/s New Data Upload : 51%|█████ | 958MB / 1.88GB, 91.8MB/s  ...nts/optim_095000_rank1.pt: 16%|█▋ | 409MB / 2.49GB  ...nts/optim_095000_rank3.pt: 16%|█▌ | 401MB / 2.49GB  ...nts/optim_095000_rank4.pt: 16%|█▋ | 408MB / 2.49GB  ...nts/optim_095000_rank5.pt: 16%|█▌ | 389MB / 2.49GB  ...nts/optim_095000_rank6.pt: 16%|█▌ | 400MB / 2.49GB  ...nts/optim_095000_rank7.pt: 16%|█▌ | 399MB / 2.49GB  ...nts/optim_095000_rank0.pt: 17%|█▋ | 430MB / 2.49GB  ...eckpoints/model_095000.pt: 5%|▍ | 115MB / 2.35GB  Processing Files (0 / 8) : 13%|█▎ | 2.95GB / 22.2GB, 285MB/s New Data Upload : 49%|████▉ | 986MB / 2.01GB, 93.7MB/s  ...nts/optim_095000_rank1.pt: 17%|█▋ | 431MB / 2.49GB  ...nts/optim_095000_rank3.pt: 17%|█▋ | 421MB / 2.49GB  ...nts/optim_095000_rank4.pt: 17%|█▋ | 430MB / 2.49GB  ...nts/optim_095000_rank5.pt: 16%|█▋ | 409MB / 2.49GB  ...nts/optim_095000_rank6.pt: 17%|█▋ | 420MB / 2.49GB  ...nts/optim_095000_rank7.pt: 17%|█▋ | 420MB / 2.49GB  ...nts/optim_095000_rank0.pt: 18%|█▊ | 451MB / 2.49GB  ...eckpoints/model_095000.pt: 5%|▌ | 118MB / 2.35GB  Processing Files (0 / 8) : 14%|█▍ | 3.10GB / 22.2GB, 299MB/s New Data Upload : 51%|█████ | 1.02GB / 2.01GB, 96.7MB/s  ...nts/optim_095000_rank1.pt: 18%|█▊ | 448MB / 2.49GB  ...nts/optim_095000_rank3.pt: 18%|█▊ | 437MB / 2.49GB  ...nts/optim_095000_rank4.pt: 18%|█▊ | 444MB / 2.49GB  ...nts/optim_095000_rank5.pt: 17%|█▋ | 429MB / 2.49GB  ...nts/optim_095000_rank6.pt: 18%|█▊ | 437MB / 2.49GB  ...nts/optim_095000_rank7.pt: 17%|█▋ | 435MB / 2.49GB  ...nts/optim_095000_rank0.pt: 19%|█▊ | 465MB / 2.49GB  ...eckpoints/model_095000.pt: 5%|▌ | 127MB / 2.35GB  Processing Files (0 / 8) : 14%|█▍ | 3.22GB / 22.2GB, 310MB/s New Data Upload : 51%|█████ | 1.06GB / 2.08GB, 99.6MB/s  ...nts/optim_095000_rank1.pt: 19%|█▉ | 475MB / 2.49GB  ...nts/optim_095000_rank3.pt: 18%|█▊ | 459MB / 2.49GB  ...nts/optim_095000_rank4.pt: 19%|█▉ | 469MB / 2.49GB  ...nts/optim_095000_rank5.pt: 18%|█▊ | 452MB / 2.49GB  ...nts/optim_095000_rank6.pt: 19%|█▊ | 460MB / 2.49GB  ...nts/optim_095000_rank7.pt: 18%|█▊ | 457MB / 2.49GB  ...nts/optim_095000_rank0.pt: 20%|█▉ | 488MB / 2.49GB  ...eckpoints/model_095000.pt: 6%|▌ | 133MB / 2.35GB  Processing Files (0 / 8) : 15%|█▌ | 3.39GB / 22.2GB, 326MB/s New Data Upload : 51%|█████▏ | 1.10GB / 2.15GB, 103MB/s  ...nts/optim_095000_rank1.pt: 20%|█▉ | 496MB / 2.49GB  ...nts/optim_095000_rank3.pt: 19%|█▉ | 480MB / 2.49GB  ...nts/optim_095000_rank4.pt: 20%|█▉ | 486MB / 2.49GB  ...nts/optim_095000_rank5.pt: 19%|█▉ | 475MB / 2.49GB  ...nts/optim_095000_rank6.pt: 19%|█▉ | 481MB / 2.49GB  ...nts/optim_095000_rank7.pt: 19%|█▉ | 477MB / 2.49GB  ...nts/optim_095000_rank0.pt: 20%|██ | 505MB / 2.49GB  ...eckpoints/model_095000.pt: 6%|▌ | 138MB / 2.35GB  Processing Files (0 / 8) : 16%|█▌ | 3.54GB / 22.2GB, 339MB/s New Data Upload : 48%|████▊ | 1.14GB / 2.35GB, 106MB/s  ...nts/optim_095000_rank1.pt: 21%|██▏ | 531MB / 2.49GB  ...nts/optim_095000_rank3.pt: 21%|██ | 512MB / 2.49GB  ...nts/optim_095000_rank4.pt: 21%|██ | 517MB / 2.49GB  ...nts/optim_095000_rank5.pt: 20%|██ | 508MB / 2.49GB  ...nts/optim_095000_rank6.pt: 21%|██ | 513MB / 2.49GB  ...nts/optim_095000_rank7.pt: 20%|██ | 509MB / 2.49GB  ...nts/optim_095000_rank0.pt: 21%|██▏ | 534MB / 2.49GB  ...eckpoints/model_095000.pt: 6%|▋ | 147MB / 2.35GB  Processing Files (0 / 8) : 17%|█▋ | 3.77GB / 22.2GB, 362MB/s New Data Upload : 48%|████▊ | 1.20GB / 2.48GB, 111MB/s  ...nts/optim_095000_rank1.pt: 22%|██▏ | 553MB / 2.49GB  ...nts/optim_095000_rank3.pt: 22%|██▏ | 535MB / 2.49GB  ...nts/optim_095000_rank4.pt: 22%|██▏ | 538MB / 2.49GB  ...nts/optim_095000_rank5.pt: 21%|██▏ | 534MB / 2.49GB  ...nts/optim_095000_rank6.pt: 22%|██▏ | 537MB / 2.49GB  ...nts/optim_095000_rank7.pt: 21%|██▏ | 531MB / 2.49GB  ...nts/optim_095000_rank0.pt: 22%|██▏ | 556MB / 2.49GB  ...eckpoints/model_095000.pt: 7%|▋ | 153MB / 2.35GB  Processing Files (0 / 8) : 18%|█▊ | 3.94GB / 22.2GB, 377MB/s New Data Upload : 48%|████▊ | 1.24GB / 2.55GB, 115MB/s  ...nts/optim_095000_rank1.pt: 24%|██▎ | 586MB / 2.49GB  ...nts/optim_095000_rank3.pt: 23%|██▎ | 567MB / 2.49GB  ...nts/optim_095000_rank4.pt: 23%|██▎ | 568MB / 2.49GB  ...nts/optim_095000_rank5.pt: 23%|██▎ | 566MB / 2.49GB  ...nts/optim_095000_rank6.pt: 23%|██▎ | 569MB / 2.49GB  ...nts/optim_095000_rank7.pt: 23%|██▎ | 562MB / 2.49GB  ...nts/optim_095000_rank0.pt: 23%|██▎ | 584MB / 2.49GB  ...eckpoints/model_095000.pt: 7%|▋ | 163MB / 2.35GB  Processing Files (0 / 8) : 19%|█▊ | 4.16GB / 22.2GB, 399MB/s New Data Upload : 51%|█████ | 1.29GB / 2.55GB, 120MB/s  ...nts/optim_095000_rank1.pt: 25%|██▍ | 618MB / 2.49GB  ...nts/optim_095000_rank3.pt: 24%|██▍ | 598MB / 2.49GB  ...nts/optim_095000_rank4.pt: 24%|██▍ | 600MB / 2.49GB  ...nts/optim_095000_rank5.pt: 24%|██▍ | 601MB / 2.49GB  ...nts/optim_095000_rank6.pt: 24%|██▍ | 601MB / 2.49GB  ...nts/optim_095000_rank7.pt: 24%|██▍ | 593MB / 2.49GB  ...nts/optim_095000_rank0.pt: 25%|██▍ | 614MB / 2.49GB  ...eckpoints/model_095000.pt: 7%|▋ | 169MB / 2.35GB  Processing Files (0 / 8) : 20%|█▉ | 4.40GB / 22.2GB, 422MB/s New Data Upload : 53%|█████▎ | 1.34GB / 2.55GB, 125MB/s  ...nts/optim_095000_rank1.pt: 26%|██▌ | 642MB / 2.49GB  ...nts/optim_095000_rank3.pt: 25%|██▍ | 619MB / 2.49GB  ...nts/optim_095000_rank4.pt: 25%|██▍ | 619MB / 2.49GB  ...nts/optim_095000_rank5.pt: 25%|██▍ | 621MB / 2.49GB  ...nts/optim_095000_rank6.pt: 25%|██▍ | 621MB / 2.49GB  ...nts/optim_095000_rank7.pt: 25%|██▍ | 614MB / 2.49GB  ...nts/optim_095000_rank0.pt: 25%|██▌ | 633MB / 2.49GB  ...eckpoints/model_095000.pt: 8%|▊ | 177MB / 2.35GB  Processing Files (0 / 8) : 20%|██ | 4.55GB / 22.2GB, 437MB/s New Data Upload : 53%|█████▎ | 1.38GB / 2.62GB, 129MB/s  ...nts/optim_095000_rank1.pt: 27%|██▋ | 663MB / 2.49GB  ...nts/optim_095000_rank3.pt: 26%|██▌ | 643MB / 2.49GB  ...nts/optim_095000_rank4.pt: 26%|██▌ | 642MB / 2.49GB  ...nts/optim_095000_rank5.pt: 26%|██▌ | 647MB / 2.49GB  ...nts/optim_095000_rank6.pt: 26%|██▌ | 645MB / 2.49GB  ...nts/optim_095000_rank7.pt: 26%|██▌ | 638MB / 2.49GB  ...nts/optim_095000_rank0.pt: 26%|██▋ | 655MB / 2.49GB  ...eckpoints/model_095000.pt: 8%|▊ | 182MB / 2.35GB  Processing Files (0 / 8) : 21%|██ | 4.72GB / 22.2GB, 453MB/s New Data Upload : 53%|█████▎ | 1.42GB / 2.68GB, 132MB/s  ...nts/optim_095000_rank1.pt: 28%|██▊ | 691MB / 2.49GB  ...nts/optim_095000_rank3.pt: 27%|██▋ | 668MB / 2.49GB  ...nts/optim_095000_rank4.pt: 27%|██▋ | 666MB / 2.49GB  ...nts/optim_095000_rank5.pt: 27%|██▋ | 671MB / 2.49GB  ...nts/optim_095000_rank6.pt: 27%|██▋ | 670MB / 2.49GB  ...nts/optim_095000_rank7.pt: 27%|██▋ | 662MB / 2.49GB  ...nts/optim_095000_rank0.pt: 27%|██▋ | 679MB / 2.49GB  ...eckpoints/model_095000.pt: 8%|▊ | 189MB / 2.35GB  Processing Files (0 / 8) : 22%|██▏ | 4.90GB / 22.2GB, 470MB/s New Data Upload : 54%|█████▍ | 1.46GB / 2.68GB, 136MB/s  ...nts/optim_095000_rank1.pt: 29%|██▊ | 714MB / 2.49GB  ...nts/optim_095000_rank3.pt: 28%|██▊ | 694MB / 2.49GB  ...nts/optim_095000_rank4.pt: 28%|██▊ | 689MB / 2.49GB  ...nts/optim_095000_rank5.pt: 28%|██▊ | 700MB / 2.49GB  ...nts/optim_095000_rank6.pt: 28%|██▊ | 696MB / 2.49GB  ...nts/optim_095000_rank7.pt: 28%|██▊ | 687MB / 2.49GB  ...nts/optim_095000_rank0.pt: 28%|██▊ | 702MB / 2.49GB  ...eckpoints/model_095000.pt: 8%|▊ | 191MB / 2.35GB  Processing Files (0 / 8) : 23%|██▎ | 5.07GB / 22.2GB, 488MB/s New Data Upload : 56%|█████▌ | 1.50GB / 2.68GB, 140MB/s  ...nts/optim_095000_rank1.pt: 30%|██▉ | 738MB / 2.49GB  ...nts/optim_095000_rank3.pt: 29%|██▉ | 717MB / 2.49GB  ...nts/optim_095000_rank4.pt: 29%|██▊ | 711MB / 2.49GB  ...nts/optim_095000_rank5.pt: 29%|██▉ | 723MB / 2.49GB  ...nts/optim_095000_rank6.pt: 29%|██▉ | 718MB / 2.49GB  ...nts/optim_095000_rank7.pt: 29%|██▊ | 710MB / 2.49GB  ...nts/optim_095000_rank0.pt: 29%|██▉ | 724MB / 2.49GB  ...eckpoints/model_095000.pt: 8%|▊ | 198MB / 2.35GB  Processing Files (0 / 8) : 24%|██▎ | 5.24GB / 22.2GB, 504MB/s New Data Upload : 56%|█████▌ | 1.54GB / 2.75GB, 144MB/s  ...nts/optim_095000_rank1.pt: 31%|███ | 760MB / 2.49GB  ...nts/optim_095000_rank3.pt: 30%|██▉ | 736MB / 2.49GB  ...nts/optim_095000_rank4.pt: 29%|██▉ | 732MB / 2.49GB  ...nts/optim_095000_rank5.pt: 30%|██▉ | 743MB / 2.49GB  ...nts/optim_095000_rank6.pt: 30%|██▉ | 738MB / 2.49GB  ...nts/optim_095000_rank7.pt: 29%|██▉ | 730MB / 2.49GB  ...nts/optim_095000_rank0.pt: 30%|██▉ | 743MB / 2.49GB  ...eckpoints/model_095000.pt: 9%|▊ | 204MB / 2.35GB  Processing Files (0 / 8) : 24%|██▍ | 5.39GB / 22.2GB, 517MB/s New Data Upload : 57%|█████▋ | 1.57GB / 2.75GB, 146MB/s  ...nts/optim_095000_rank1.pt: 31%|███▏ | 781MB / 2.49GB  ...nts/optim_095000_rank3.pt: 30%|███ | 754MB / 2.49GB  ...nts/optim_095000_rank4.pt: 30%|███ | 751MB / 2.49GB  ...nts/optim_095000_rank5.pt: 31%|███ | 760MB / 2.49GB  ...nts/optim_095000_rank6.pt: 30%|███ | 754MB / 2.49GB  ...nts/optim_095000_rank7.pt: 30%|███ | 748MB / 2.49GB  ...nts/optim_095000_rank0.pt: 31%|███ | 761MB / 2.49GB  ...eckpoints/model_095000.pt: 9%|▉ | 211MB / 2.35GB  Processing Files (0 / 8) : 25%|██▍ | 5.52GB / 22.2GB, 529MB/s New Data Upload : 58%|█████▊ | 1.60GB / 2.75GB, 149MB/s  ...nts/optim_095000_rank1.pt: 32%|███▏ | 797MB / 2.49GB  ...nts/optim_095000_rank3.pt: 31%|███ | 772MB / 2.49GB  ...nts/optim_095000_rank4.pt: 31%|███ | 766MB / 2.49GB  ...nts/optim_095000_rank5.pt: 31%|███▏ | 778MB / 2.49GB  ...nts/optim_095000_rank6.pt: 31%|███ | 771MB / 2.49GB  ...nts/optim_095000_rank7.pt: 31%|███ | 762MB / 2.49GB  ...nts/optim_095000_rank0.pt: 31%|███ | 776MB / 2.49GB  ...eckpoints/model_095000.pt: 9%|▉ | 213MB / 2.35GB  Processing Files (0 / 8) : 25%|██▌ | 5.64GB / 22.2GB, 539MB/s New Data Upload : 59%|█████▉ | 1.63GB / 2.75GB, 150MB/s  ...nts/optim_095000_rank1.pt: 33%|███▎ | 818MB / 2.49GB  ...nts/optim_095000_rank3.pt: 32%|███▏ | 791MB / 2.49GB  ...nts/optim_095000_rank4.pt: 32%|███▏ | 784MB / 2.49GB  ...nts/optim_095000_rank5.pt: 32%|███▏ | 797MB / 2.49GB  ...nts/optim_095000_rank6.pt: 32%|███▏ | 790MB / 2.49GB  ...nts/optim_095000_rank7.pt: 31%|███▏ | 781MB / 2.49GB  ...nts/optim_095000_rank0.pt: 32%|███▏ | 794MB / 2.49GB  ...eckpoints/model_095000.pt: 9%|▉ | 220MB / 2.35GB  Processing Files (0 / 8) : 26%|██▌ | 5.77GB / 22.2GB, 550MB/s New Data Upload : 61%|██████ | 1.67GB / 2.75GB, 151MB/s  ...nts/optim_095000_rank1.pt: 34%|███▎ | 834MB / 2.49GB  ...nts/optim_095000_rank3.pt: 33%|███▎ | 810MB / 2.49GB  ...nts/optim_095000_rank4.pt: 32%|███▏ | 797MB / 2.49GB  ...nts/optim_095000_rank5.pt: 33%|███▎ | 811MB / 2.49GB  ...nts/optim_095000_rank6.pt: 32%|███▏ | 803MB / 2.49GB  ...nts/optim_095000_rank7.pt: 32%|███▏ | 799MB / 2.49GB  ...nts/optim_095000_rank0.pt: 32%|███▏ | 808MB / 2.49GB  ...eckpoints/model_095000.pt: 10%|▉ | 227MB / 2.35GB  Processing Files (0 / 8) : 26%|██▋ | 5.89GB / 22.2GB, 558MB/s New Data Upload : 60%|██████ | 1.70GB / 2.82GB, 152MB/s  ...nts/optim_095000_rank1.pt: 35%|███▍ | 859MB / 2.49GB  ...nts/optim_095000_rank3.pt: 34%|███▎ | 835MB / 2.49GB  ...nts/optim_095000_rank4.pt: 33%|███▎ | 823MB / 2.49GB  ...nts/optim_095000_rank5.pt: 34%|███▎ | 833MB / 2.49GB  ...nts/optim_095000_rank6.pt: 33%|███▎ | 829MB / 2.49GB  ...nts/optim_095000_rank7.pt: 33%|███▎ | 821MB / 2.49GB  ...nts/optim_095000_rank0.pt: 34%|███▎ | 833MB / 2.49GB  ...eckpoints/model_095000.pt: 10%|▉ | 230MB / 2.35GB  Processing Files (0 / 8) : 27%|██▋ | 6.06GB / 22.2GB, 573MB/s New Data Upload : 62%|██████▏ | 1.74GB / 2.82GB, 155MB/s  ...nts/optim_095000_rank1.pt: 35%|███▌ | 873MB / 2.49GB  ...nts/optim_095000_rank3.pt: 34%|███▍ | 849MB / 2.49GB  ...nts/optim_095000_rank4.pt: 34%|███▎ | 836MB / 2.49GB  ...nts/optim_095000_rank5.pt: 34%|███▍ | 847MB / 2.49GB  ...nts/optim_095000_rank6.pt: 34%|███▍ | 841MB / 2.49GB  ...nts/optim_095000_rank7.pt: 34%|███▎ | 838MB / 2.49GB  ...nts/optim_095000_rank0.pt: 34%|███▍ | 843MB / 2.49GB  ...eckpoints/model_095000.pt: 10%|█ | 236MB / 2.35GB  Processing Files (0 / 8) : 28%|██▊ | 6.16GB / 22.2GB, 580MB/s New Data Upload : 61%|██████▏ | 1.77GB / 2.88GB, 155MB/s  ...nts/optim_095000_rank1.pt: 36%|███▌ | 891MB / 2.49GB  ...nts/optim_095000_rank3.pt: 35%|███▍ | 868MB / 2.49GB  ...nts/optim_095000_rank4.pt: 34%|███▍ | 851MB / 2.49GB  ...nts/optim_095000_rank5.pt: 35%|███▍ | 865MB / 2.49GB  ...nts/optim_095000_rank6.pt: 35%|███▍ | 860MB / 2.49GB  ...nts/optim_095000_rank7.pt: 35%|███▍ | 863MB / 2.49GB  ...nts/optim_095000_rank0.pt: 35%|███▍ | 861MB / 2.49GB  ...eckpoints/model_095000.pt: 10%|█ | 240MB / 2.35GB  Processing Files (0 / 8) : 28%|██▊ | 6.30GB / 22.2GB, 591MB/s New Data Upload : 63%|██████▎ | 1.82GB / 2.88GB, 158MB/s  ...nts/optim_095000_rank1.pt: 36%|███▋ | 904MB / 2.49GB  ...nts/optim_095000_rank3.pt: 35%|███▌ | 881MB / 2.49GB  ...nts/optim_095000_rank4.pt: 35%|███▍ | 860MB / 2.49GB  ...nts/optim_095000_rank5.pt: 35%|███▌ | 877MB / 2.49GB  ...nts/optim_095000_rank6.pt: 35%|███▌ | 873MB / 2.49GB  ...nts/optim_095000_rank7.pt: 35%|███▌ | 874MB / 2.49GB  ...nts/optim_095000_rank0.pt: 35%|███▌ | 875MB / 2.49GB  ...eckpoints/model_095000.pt: 10%|█ | 246MB / 2.35GB  Processing Files (0 / 8) : 29%|██▊ | 6.39GB / 22.2GB, 597MB/s New Data Upload : 60%|█████▉ | 1.85GB / 3.09GB, 159MB/s  ...nts/optim_095000_rank1.pt: 37%|███▋ | 917MB / 2.49GB  ...nts/optim_095000_rank3.pt: 36%|███▌ | 896MB / 2.49GB  ...nts/optim_095000_rank4.pt: 35%|███▌ | 872MB / 2.49GB  ...nts/optim_095000_rank5.pt: 36%|███▌ | 892MB / 2.49GB  ...nts/optim_095000_rank6.pt: 36%|███▌ | 888MB / 2.49GB  ...nts/optim_095000_rank7.pt: 36%|███▌ | 892MB / 2.49GB  ...nts/optim_095000_rank0.pt: 36%|███▌ | 890MB / 2.49GB  ...eckpoints/model_095000.pt: 11%|█ | 249MB / 2.35GB  Processing Files (0 / 8) : 29%|██▉ | 6.50GB / 22.2GB, 606MB/s New Data Upload : 61%|██████ | 1.88GB / 3.09GB, 160MB/s  ...nts/optim_095000_rank1.pt: 38%|███▊ | 933MB / 2.49GB  ...nts/optim_095000_rank3.pt: 37%|███▋ | 910MB / 2.49GB  ...nts/optim_095000_rank4.pt: 36%|███▌ | 886MB / 2.49GB  ...nts/optim_095000_rank5.pt: 37%|███▋ | 910MB / 2.49GB  ...nts/optim_095000_rank6.pt: 36%|███▋ | 904MB / 2.49GB  ...nts/optim_095000_rank7.pt: 36%|███▋ | 904MB / 2.49GB  ...nts/optim_095000_rank0.pt: 36%|███▋ | 906MB / 2.49GB  ...eckpoints/model_095000.pt: 11%|█ | 253MB / 2.35GB  Processing Files (0 / 8) : 30%|██▉ | 6.60GB / 22.2GB, 616MB/s New Data Upload : 61%|██████ | 1.91GB / 3.15GB, 163MB/s  ...nts/optim_095000_rank1.pt: 38%|███▊ | 953MB / 2.49GB  ...nts/optim_095000_rank3.pt: 37%|███▋ | 931MB / 2.49GB  ...nts/optim_095000_rank4.pt: 36%|███▋ | 904MB / 2.49GB  ...nts/optim_095000_rank5.pt: 38%|███▊ | 934MB / 2.49GB  ...nts/optim_095000_rank6.pt: 37%|███▋ | 926MB / 2.49GB  ...nts/optim_095000_rank7.pt: 37%|███▋ | 926MB / 2.49GB  ...nts/optim_095000_rank0.pt: 37%|███▋ | 927MB / 2.49GB  ...eckpoints/model_095000.pt: 11%|█ | 257MB / 2.35GB  Processing Files (0 / 8) : 30%|███ | 6.76GB / 22.2GB, 629MB/s New Data Upload : 61%|██████ | 1.96GB / 3.22GB, 166MB/s  ...nts/optim_095000_rank1.pt: 39%|███▉ | 971MB / 2.49GB  ...nts/optim_095000_rank3.pt: 38%|███▊ | 951MB / 2.49GB  ...nts/optim_095000_rank4.pt: 37%|███▋ | 923MB / 2.49GB  ...nts/optim_095000_rank5.pt: 38%|███▊ | 950MB / 2.49GB  ...nts/optim_095000_rank6.pt: 38%|███▊ | 945MB / 2.49GB  ...nts/optim_095000_rank7.pt: 38%|███▊ | 944MB / 2.49GB  ...nts/optim_095000_rank0.pt: 38%|███▊ | 944MB / 2.49GB  ...eckpoints/model_095000.pt: 11%|█ | 262MB / 2.35GB  Processing Files (0 / 8) : 31%|███ | 6.89GB / 22.2GB, 641MB/s New Data Upload : 62%|██████▏ | 1.99GB / 3.22GB, 168MB/s  ...nts/optim_095000_rank1.pt: 40%|████ | 999MB / 2.49GB  ...nts/optim_095000_rank3.pt: 39%|███▉ | 977MB / 2.49GB  ...nts/optim_095000_rank4.pt: 38%|███▊ | 949MB / 2.49GB  ...nts/optim_095000_rank5.pt: 39%|███▉ | 976MB / 2.49GB  ...nts/optim_095000_rank6.pt: 39%|███▉ | 971MB / 2.49GB  ...nts/optim_095000_rank7.pt: 39%|███▉ | 976MB / 2.49GB  ...nts/optim_095000_rank0.pt: 39%|███▉ | 972MB / 2.49GB  ...eckpoints/model_095000.pt: 11%|█ | 264MB / 2.35GB  Processing Files (0 / 8) : 32%|███▏ | 7.08GB / 22.2GB, 655MB/s New Data Upload : 62%|██████▏ | 2.04GB / 3.29GB, 170MB/s  ...nts/optim_095000_rank1.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_095000_rank3.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_095000_rank4.pt: 39%|███▉ | 978MB / 2.49GB  ...nts/optim_095000_rank5.pt: 40%|████ | 1.01GB / 2.49GB  ...nts/optim_095000_rank6.pt: 40%|████ | 1.00GB / 2.49GB  ...nts/optim_095000_rank7.pt: 41%|████ | 1.01GB / 2.49GB  ...nts/optim_095000_rank0.pt: 40%|████ | 1.00GB / 2.49GB  ...eckpoints/model_095000.pt: 11%|█▏ | 268MB / 2.35GB  Processing Files (0 / 8) : 33%|███▎ | 7.30GB / 22.2GB, 670MB/s New Data Upload : 64%|██████▎ | 2.09GB / 3.29GB, 173MB/s  ...nts/optim_095000_rank1.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_095000_rank3.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_095000_rank4.pt: 40%|████ | 998MB / 2.49GB  ...nts/optim_095000_rank5.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_095000_rank6.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_095000_rank7.pt: 41%|████▏ | 1.03GB / 2.49GB  ...nts/optim_095000_rank0.pt: 41%|████ | 1.02GB / 2.49GB  ...eckpoints/model_095000.pt: 12%|█▏ | 270MB / 2.35GB  Processing Files (0 / 8) : 34%|███▎ | 7.45GB / 22.2GB, 680MB/s New Data Upload : 63%|██████▎ | 2.13GB / 3.35GB, 174MB/s  ...nts/optim_095000_rank1.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_095000_rank3.pt: 42%|████▏ | 1.06GB / 2.49GB  ...nts/optim_095000_rank4.pt: 41%|████ | 1.02GB / 2.49GB  ...nts/optim_095000_rank5.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_095000_rank6.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_095000_rank7.pt: 43%|████▎ | 1.06GB / 2.49GB  ...nts/optim_095000_rank0.pt: 42%|████▏ | 1.05GB / 2.49GB  ...eckpoints/model_095000.pt: 12%|█▏ | 274MB / 2.35GB  Processing Files (0 / 8) : 34%|███▍ | 7.64GB / 22.2GB, 691MB/s New Data Upload : 61%|██████ | 2.17GB / 3.55GB, 176MB/s  ...nts/optim_095000_rank1.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_095000_rank3.pt: 43%|████▎ | 1.08GB / 2.49GB  ...nts/optim_095000_rank4.pt: 42%|████▏ | 1.05GB / 2.49GB  ...nts/optim_095000_rank5.pt: 44%|████▎ | 1.08GB / 2.49GB  ...nts/optim_095000_rank6.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_095000_rank7.pt: 44%|████▎ | 1.09GB / 2.49GB  ...nts/optim_095000_rank0.pt: 43%|████▎ | 1.07GB / 2.49GB  ...eckpoints/model_095000.pt: 12%|█▏ | 279MB / 2.35GB  Processing Files (0 / 8) : 35%|███▌ | 7.81GB / 22.2GB, 703MB/s New Data Upload : 61%|██████ | 2.21GB / 3.62GB, 178MB/s  ...nts/optim_095000_rank1.pt: 45%|████▌ | 1.12GB / 2.49GB  ...nts/optim_095000_rank3.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_095000_rank4.pt: 43%|████▎ | 1.07GB / 2.49GB  ...nts/optim_095000_rank5.pt: 45%|████▍ | 1.11GB / 2.49GB  ...nts/optim_095000_rank6.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_095000_rank7.pt: 45%|████▍ | 1.12GB / 2.49GB  ...nts/optim_095000_rank0.pt: 44%|████▍ | 1.09GB / 2.49GB  ...eckpoints/model_095000.pt: 12%|█▏ | 281MB / 2.35GB  Processing Files (0 / 8) : 36%|███▌ | 7.99GB / 22.2GB, 713MB/s New Data Upload : 60%|██████ | 2.26GB / 3.76GB, 180MB/s  ...nts/optim_095000_rank1.pt: 47%|████▋ | 1.16GB / 2.49GB  ...nts/optim_095000_rank3.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_095000_rank4.pt: 44%|████▍ | 1.10GB / 2.49GB  ...nts/optim_095000_rank5.pt: 46%|████▌ | 1.14GB / 2.49GB  ...nts/optim_095000_rank6.pt: 45%|████▌ | 1.13GB / 2.49GB  ...nts/optim_095000_rank7.pt: 46%|████▋ | 1.15GB / 2.49GB  ...nts/optim_095000_rank0.pt: 45%|████▌ | 1.12GB / 2.49GB  ...eckpoints/model_095000.pt: 12%|█▏ | 287MB / 2.35GB  Processing Files (0 / 8) : 37%|███▋ | 8.24GB / 22.2GB, 728MB/s New Data Upload : 60%|██████ | 2.31GB / 3.82GB, 182MB/s  ...nts/optim_095000_rank1.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_095000_rank3.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_095000_rank4.pt: 45%|████▌ | 1.12GB / 2.49GB  ...nts/optim_095000_rank5.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_095000_rank6.pt: 46%|████▋ | 1.15GB / 2.49GB  ...nts/optim_095000_rank7.pt: 47%|████▋ | 1.18GB / 2.49GB  ...nts/optim_095000_rank0.pt: 46%|████▌ | 1.15GB / 2.49GB  ...eckpoints/model_095000.pt: 12%|█▏ | 292MB / 2.35GB  Processing Files (0 / 8) : 38%|███▊ | 8.42GB / 22.2GB, 738MB/s New Data Upload : 60%|██████ | 2.35GB / 3.89GB, 183MB/s  ...nts/optim_095000_rank1.pt: 49%|████▉ | 1.21GB / 2.49GB  ...nts/optim_095000_rank3.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_095000_rank4.pt: 46%|████▋ | 1.15GB / 2.49GB  ...nts/optim_095000_rank5.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_095000_rank6.pt: 47%|████▋ | 1.18GB / 2.49GB  ...nts/optim_095000_rank7.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_095000_rank0.pt: 47%|████▋ | 1.17GB / 2.49GB  ...eckpoints/model_095000.pt: 13%|█▎ | 301MB / 2.35GB  Processing Files (0 / 8) : 39%|███▊ | 8.62GB / 22.2GB, 741MB/s New Data Upload : 61%|██████ | 2.40GB / 3.96GB, 184MB/s  ...nts/optim_095000_rank1.pt: 50%|████▉ | 1.23GB / 2.49GB  ...nts/optim_095000_rank3.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_095000_rank4.pt: 47%|████▋ | 1.17GB / 2.49GB  ...nts/optim_095000_rank5.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_095000_rank6.pt: 48%|████▊ | 1.20GB / 2.49GB  ...nts/optim_095000_rank7.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_095000_rank0.pt: 48%|████▊ | 1.19GB / 2.49GB  ...eckpoints/model_095000.pt: 13%|█▎ | 312MB / 2.35GB  Processing Files (0 / 8) : 39%|███▉ | 8.75GB / 22.2GB, 743MB/s New Data Upload : 62%|██████▏ | 2.44GB / 3.96GB, 184MB/s  ...nts/optim_095000_rank1.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_095000_rank3.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_095000_rank4.pt: 48%|████▊ | 1.19GB / 2.49GB  ...nts/optim_095000_rank5.pt: 50%|█████ | 1.24GB / 2.49GB  ...nts/optim_095000_rank6.pt: 49%|████▉ | 1.22GB / 2.49GB  ...nts/optim_095000_rank7.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_095000_rank0.pt: 49%|████▉ | 1.22GB / 2.49GB  ...eckpoints/model_095000.pt: 14%|█▎ | 322MB / 2.35GB  Processing Files (0 / 8) : 40%|████ | 8.93GB / 22.2GB, 744MB/s New Data Upload : 63%|██████▎ | 2.48GB / 3.96GB, 185MB/s  ...nts/optim_095000_rank1.pt: 52%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_095000_rank3.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_095000_rank4.pt: 49%|████▊ | 1.21GB / 2.49GB  ...nts/optim_095000_rank5.pt: 51%|█████ | 1.27GB / 2.49GB  ...nts/optim_095000_rank6.pt: 50%|████▉ | 1.24GB / 2.49GB  ...nts/optim_095000_rank7.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_095000_rank0.pt: 50%|████▉ | 1.24GB / 2.49GB  ...eckpoints/model_095000.pt: 14%|█▍ | 327MB / 2.35GB  Processing Files (0 / 8) : 41%|████ | 9.09GB / 22.2GB, 736MB/s New Data Upload : 63%|██████▎ | 2.53GB / 4.02GB, 185MB/s  ...nts/optim_095000_rank1.pt: 52%|█████▏ | 1.30GB / 2.49GB  ...nts/optim_095000_rank3.pt: 52%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_095000_rank4.pt: 50%|████▉ | 1.23GB / 2.49GB  ...nts/optim_095000_rank5.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_095000_rank6.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_095000_rank7.pt: 52%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_095000_rank0.pt: 51%|█████ | 1.26GB / 2.49GB  ...eckpoints/model_095000.pt: 14%|█▍ | 336MB / 2.35GB  Processing Files (0 / 8) : 42%|████▏ | 9.24GB / 22.2GB, 742MB/s New Data Upload : 63%|██████▎ | 2.57GB / 4.09GB, 187MB/s  ...nts/optim_095000_rank1.pt: 53%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_095000_rank3.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_095000_rank4.pt: 51%|█████ | 1.26GB / 2.49GB  ...nts/optim_095000_rank5.pt: 53%|█████▎ | 1.32GB / 2.49GB  ...nts/optim_095000_rank6.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_095000_rank7.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_095000_rank0.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...eckpoints/model_095000.pt: 15%|█▍ | 345MB / 2.35GB  Processing Files (0 / 8) : 42%|████▏ | 9.44GB / 22.2GB, 747MB/s New Data Upload : 63%|██████▎ | 2.62GB / 4.16GB, 188MB/s  ...nts/optim_095000_rank1.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_095000_rank3.pt: 54%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_095000_rank4.pt: 51%|█████▏ | 1.28GB / 2.49GB  ...nts/optim_095000_rank5.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_095000_rank6.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_095000_rank7.pt: 53%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_095000_rank0.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...eckpoints/model_095000.pt: 15%|█▌ | 353MB / 2.35GB  Processing Files (0 / 8) : 43%|████▎ | 9.60GB / 22.2GB, 746MB/s New Data Upload : 63%|██████▎ | 2.67GB / 4.23GB, 189MB/s  ...nts/optim_095000_rank1.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_095000_rank3.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_095000_rank4.pt: 52%|█████▏ | 1.29GB / 2.49GB  ...nts/optim_095000_rank5.pt: 55%|█████▍ | 1.37GB / 2.49GB  ...nts/optim_095000_rank6.pt: 53%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_095000_rank7.pt: 54%|█████▍ | 1.34GB / 2.49GB  ...nts/optim_095000_rank0.pt: 54%|█████▎ | 1.34GB / 2.49GB  ...eckpoints/model_095000.pt: 15%|█▌ | 359MB / 2.35GB  Processing Files (0 / 8) : 44%|████▍ | 9.74GB / 22.2GB, 752MB/s New Data Upload : 64%|██████▍ | 2.72GB / 4.23GB, 192MB/s  ...nts/optim_095000_rank1.pt: 56%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_095000_rank3.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_095000_rank4.pt: 53%|█████▎ | 1.31GB / 2.49GB  ...nts/optim_095000_rank5.pt: 56%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_095000_rank6.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_095000_rank7.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...nts/optim_095000_rank0.pt: 55%|█████▍ | 1.36GB / 2.49GB  ...eckpoints/model_095000.pt: 16%|█▌ | 364MB / 2.35GB  Processing Files (0 / 8) : 44%|████▍ | 9.87GB / 22.2GB, 753MB/s New Data Upload : 65%|██████▌ | 2.75GB / 4.23GB, 193MB/s  ...nts/optim_095000_rank1.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_095000_rank3.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_095000_rank4.pt: 54%|█████▎ | 1.33GB / 2.49GB  ...nts/optim_095000_rank5.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_095000_rank6.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_095000_rank7.pt: 56%|█████▌ | 1.39GB / 2.49GB  ...nts/optim_095000_rank0.pt: 56%|█████▌ | 1.38GB / 2.49GB  ...eckpoints/model_095000.pt: 16%|█▌ | 371MB / 2.35GB  Processing Files (0 / 8) : 45%|████▌ | 10.0GB / 22.2GB, 758MB/s New Data Upload : 65%|██████▌ | 2.81GB / 4.29GB, 195MB/s  ...nts/optim_095000_rank1.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_095000_rank3.pt: 57%|█████▋ | 1.41GB / 2.49GB  ...nts/optim_095000_rank4.pt: 54%|█████▍ | 1.35GB / 2.49GB  ...nts/optim_095000_rank5.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_095000_rank6.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_095000_rank7.pt: 57%|█████▋ | 1.42GB / 2.49GB  ...nts/optim_095000_rank0.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...eckpoints/model_095000.pt: 16%|█▌ | 382MB / 2.35GB  Processing Files (0 / 8) : 46%|████▌ | 10.2GB / 22.2GB, 765MB/s New Data Upload : 65%|██████▌ | 2.85GB / 4.36GB, 197MB/s  ...nts/optim_095000_rank1.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_095000_rank3.pt: 58%|█████▊ | 1.43GB / 2.49GB  ...nts/optim_095000_rank4.pt: 55%|█████▌ | 1.38GB / 2.49GB  ...nts/optim_095000_rank5.pt: 58%|█████▊ | 1.44GB / 2.49GB  ...nts/optim_095000_rank6.pt: 58%|█████▊ | 1.43GB / 2.49GB  ...nts/optim_095000_rank7.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_095000_rank0.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...eckpoints/model_095000.pt: 17%|█▋ | 388MB / 2.35GB  Processing Files (0 / 8) : 47%|████▋ | 10.4GB / 22.2GB, 771MB/s New Data Upload : 66%|██████▌ | 2.90GB / 4.43GB, 199MB/s  ...nts/optim_095000_rank1.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_095000_rank3.pt: 59%|█████▊ | 1.46GB / 2.49GB  ...nts/optim_095000_rank4.pt: 56%|█████▋ | 1.40GB / 2.49GB  ...nts/optim_095000_rank5.pt: 59%|█████▉ | 1.46GB / 2.49GB  ...nts/optim_095000_rank6.pt: 59%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_095000_rank7.pt: 59%|█████▉ | 1.47GB / 2.49GB  ...nts/optim_095000_rank0.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...eckpoints/model_095000.pt: 17%|█▋ | 397MB / 2.35GB  Processing Files (0 / 8) : 48%|████▊ | 10.6GB / 22.2GB, 780MB/s New Data Upload : 64%|██████▎ | 2.95GB / 4.63GB, 202MB/s  ...nts/optim_095000_rank1.pt: 60%|██████ | 1.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_095000_rank4.pt: 57%|█████▋ | 1.43GB / 2.49GB  ...nts/optim_095000_rank5.pt: 60%|█████▉ | 1.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_095000_rank7.pt: 60%|██████ | 1.50GB / 2.49GB  ...nts/optim_095000_rank0.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...eckpoints/model_095000.pt: 17%|█▋ | 403MB / 2.35GB  Processing Files (0 / 8) : 48%|████▊ | 10.8GB / 22.2GB, 792MB/s New Data Upload : 64%|██████▍ | 3.00GB / 4.69GB, 205MB/s  ...nts/optim_095000_rank1.pt: 61%|██████ | 1.52GB / 2.49GB  ...nts/optim_095000_rank3.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_095000_rank4.pt: 58%|█████▊ | 1.45GB / 2.49GB  ...nts/optim_095000_rank5.pt: 61%|██████ | 1.52GB / 2.49GB  ...nts/optim_095000_rank6.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_095000_rank7.pt: 62%|██████▏ | 1.53GB / 2.49GB  ...nts/optim_095000_rank0.pt: 61%|██████ | 1.50GB / 2.49GB  ...eckpoints/model_095000.pt: 17%|█▋ | 411MB / 2.35GB  Processing Files (0 / 8) : 49%|████▉ | 11.0GB / 22.2GB, 802MB/s New Data Upload : 65%|██████▌ | 3.05GB / 4.69GB, 208MB/s  ...nts/optim_095000_rank1.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_095000_rank3.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_095000_rank4.pt: 60%|█████▉ | 1.48GB / 2.49GB  ...nts/optim_095000_rank5.pt: 62%|██████▏ | 1.55GB / 2.49GB  ...nts/optim_095000_rank6.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_095000_rank7.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...nts/optim_095000_rank0.pt: 61%|██████▏ | 1.53GB / 2.49GB  ...eckpoints/model_095000.pt: 18%|█▊ | 415MB / 2.35GB  Processing Files (0 / 8) : 50%|█████ | 11.1GB / 22.2GB, 813MB/s New Data Upload : 66%|██████▌ | 3.11GB / 4.69GB, 211MB/s  ...nts/optim_095000_rank2.pt: 3%|▎ | 63.9MB / 2.49GB  ...nts/optim_095000_rank1.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_095000_rank3.pt: 63%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_095000_rank4.pt: 61%|██████ | 1.51GB / 2.49GB  ...nts/optim_095000_rank5.pt: 64%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_095000_rank6.pt: 64%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_095000_rank7.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_095000_rank0.pt: 63%|██████▎ | 1.56GB / 2.49GB  ...eckpoints/model_095000.pt: 18%|█▊ | 424MB / 2.35GB  ...nts/optim_095000_rank2.pt: 3%|▎ | 63.9MB / 2.49GB  Processing Files (0 / 9) : 52%|█████▏ | 11.5GB / 22.2GB, 835MB/s New Data Upload : 68%|██████▊ | 3.18GB / 4.69GB, 215MB/s  ...nts/optim_095000_rank1.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...nts/optim_095000_rank3.pt: 64%|██████▍ | 1.60GB / 2.49GB  ...nts/optim_095000_rank4.pt: 62%|██████▏ | 1.54GB / 2.49GB  ...nts/optim_095000_rank5.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_095000_rank6.pt: 65%|██████▍ | 1.61GB / 2.49GB  ...nts/optim_095000_rank7.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_095000_rank0.pt: 64%|██████▍ | 1.59GB / 2.49GB  ...eckpoints/model_095000.pt: 18%|█▊ | 426MB / 2.35GB  ...nts/optim_095000_rank2.pt: 6%|▌ | 153MB / 2.49GB  Processing Files (0 / 9) : 53%|█████▎ | 11.7GB / 22.2GB, 846MB/s New Data Upload : 69%|██████▊ | 3.22GB / 4.69GB, 216MB/s  ...nts/optim_095000_rank1.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_095000_rank3.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_095000_rank4.pt: 63%|██████▎ | 1.57GB / 2.49GB  ...nts/optim_095000_rank5.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_095000_rank6.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_095000_rank7.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_095000_rank0.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...eckpoints/model_095000.pt: 19%|█▊ | 438MB / 2.35GB  ...nts/optim_095000_rank2.pt: 11%|█ | 265MB / 2.49GB  Processing Files (0 / 9) : 54%|█████▍ | 12.1GB / 22.2GB, 869MB/s New Data Upload : 69%|██████▉ | 3.28GB / 4.74GB, 218MB/s  ...nts/optim_095000_rank1.pt: 66%|██████▌ | 1.64GB / 2.49GB  ...nts/optim_095000_rank3.pt: 66%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_095000_rank4.pt: 64%|██████▎ | 1.58GB / 2.49GB  ...nts/optim_095000_rank5.pt: 67%|██████▋ | 1.66GB / 2.49GB  ...nts/optim_095000_rank6.pt: 67%|██████▋ | 1.65GB / 2.49GB  ...nts/optim_095000_rank7.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_095000_rank0.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...eckpoints/model_095000.pt: 19%|█▉ | 445MB / 2.35GB  ...nts/optim_095000_rank2.pt: 13%|█▎ | 329MB / 2.49GB  Processing Files (0 / 9) : 55%|█████▌ | 12.3GB / 22.2GB, 869MB/s New Data Upload : 70%|██████▉ | 3.32GB / 4.74GB, 217MB/s  ...nts/optim_095000_rank1.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_095000_rank3.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_095000_rank4.pt: 65%|██████▌ | 1.62GB / 2.49GB  ...nts/optim_095000_rank5.pt: 68%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_095000_rank6.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_095000_rank7.pt: 69%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_095000_rank0.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...eckpoints/model_095000.pt: 19%|█▉ | 453MB / 2.35GB  ...nts/optim_095000_rank2.pt: 18%|█▊ | 447MB / 2.49GB  Processing Files (0 / 9) : 57%|█████▋ | 12.6GB / 22.2GB, 891MB/s New Data Upload : 71%|███████ | 3.38GB / 4.74GB, 220MB/s  ...nts/optim_095000_rank1.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_095000_rank3.pt: 69%|██████▊ | 1.70GB / 2.49GB  ...nts/optim_095000_rank4.pt: 66%|██████▌ | 1.63GB / 2.49GB  ...nts/optim_095000_rank5.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_095000_rank6.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_095000_rank7.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_095000_rank0.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...eckpoints/model_095000.pt: 20%|█▉ | 461MB / 2.35GB  ...nts/optim_095000_rank2.pt: 22%|██▏ | 549MB / 2.49GB  Processing Files (0 / 9) : 58%|█████▊ | 12.9GB / 22.2GB, 891MB/s New Data Upload : 72%|███████▏ | 3.41GB / 4.74GB, 217MB/s  ...nts/optim_095000_rank1.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...nts/optim_095000_rank3.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_095000_rank4.pt: 67%|██████▋ | 1.67GB / 2.49GB  ...nts/optim_095000_rank5.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_095000_rank6.pt: 70%|███████ | 1.74GB / 2.49GB  ...nts/optim_095000_rank7.pt: 70%|███████ | 1.75GB / 2.49GB  ...nts/optim_095000_rank0.pt: 69%|██████▉ | 1.71GB / 2.49GB  ...eckpoints/model_095000.pt: 20%|█▉ | 467MB / 2.35GB  ...nts/optim_095000_rank2.pt: 32%|███▏ | 786MB / 2.49GB  Processing Files (0 / 9) : 60%|█████▉ | 13.3GB / 22.2GB, 919MB/s New Data Upload : 72%|███████▏ | 3.47GB / 4.81GB, 219MB/s  ...nts/optim_095000_rank1.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_095000_rank3.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_095000_rank4.pt: 68%|██████▊ | 1.69GB / 2.49GB  ...nts/optim_095000_rank5.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_095000_rank6.pt: 71%|███████ | 1.77GB / 2.49GB  ...nts/optim_095000_rank7.pt: 71%|███████▏ | 1.77GB / 2.49GB  ...nts/optim_095000_rank0.pt: 70%|██████▉ | 1.73GB / 2.49GB  ...eckpoints/model_095000.pt: 20%|██ | 474MB / 2.35GB  ...nts/optim_095000_rank2.pt: 35%|███▍ | 867MB / 2.49GB  Processing Files (0 / 9) : 61%|██████ | 13.6GB / 22.2GB, 923MB/s New Data Upload : 72%|███████▏ | 3.51GB / 4.88GB, 218MB/s  ...nts/optim_095000_rank1.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_095000_rank3.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_095000_rank4.pt: 69%|██████▉ | 1.72GB / 2.49GB  ...nts/optim_095000_rank5.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_095000_rank6.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_095000_rank7.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_095000_rank0.pt: 71%|███████ | 1.76GB / 2.49GB  ...eckpoints/model_095000.pt: 20%|██ | 479MB / 2.35GB  ...nts/optim_095000_rank2.pt: 38%|███▊ | 938MB / 2.49GB  Processing Files (0 / 9) : 62%|██████▏ | 13.8GB / 22.2GB, 924MB/s New Data Upload : 73%|███████▎ | 3.56GB / 4.88GB, 217MB/s  ...nts/optim_095000_rank1.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...nts/optim_095000_rank3.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_095000_rank4.pt: 70%|██████▉ | 1.74GB / 2.49GB  ...nts/optim_095000_rank5.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_095000_rank6.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_095000_rank7.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_095000_rank0.pt: 72%|███████▏ | 1.78GB / 2.49GB  ...eckpoints/model_095000.pt: 21%|██ | 486MB / 2.35GB  ...nts/optim_095000_rank2.pt: 44%|████▍ | 1.10GB / 2.49GB  Processing Files (0 / 9) : 64%|██████▎ | 14.1GB / 22.2GB, 939MB/s New Data Upload : 74%|███████▎ | 3.59GB / 4.88GB, 217MB/s  ...nts/optim_095000_rank1.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...nts/optim_095000_rank3.pt: 74%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_095000_rank4.pt: 71%|███████ | 1.76GB / 2.49GB  ...nts/optim_095000_rank5.pt: 74%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_095000_rank6.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_095000_rank7.pt: 74%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_095000_rank0.pt: 72%|███████▏ | 1.80GB / 2.49GB  ...eckpoints/model_095000.pt: 21%|██ | 494MB / 2.35GB  ...nts/optim_095000_rank2.pt: 47%|████▋ | 1.18GB / 2.49GB  Processing Files (0 / 9) : 65%|██████▍ | 14.4GB / 22.2GB, 946MB/s New Data Upload : 75%|███████▍ | 3.64GB / 4.88GB, 218MB/s  ...nts/optim_095000_rank1.pt: 73%|███████▎ | 1.82GB / 2.49GB  ...nts/optim_095000_rank3.pt: 75%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_095000_rank4.pt: 72%|███████▏ | 1.79GB / 2.49GB  ...nts/optim_095000_rank5.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_095000_rank6.pt: 75%|███████▌ | 1.86GB / 2.49GB  ...nts/optim_095000_rank7.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_095000_rank0.pt: 73%|███████▎ | 1.83GB / 2.49GB  ...eckpoints/model_095000.pt: 21%|██▏ | 501MB / 2.35GB  ...nts/optim_095000_rank2.pt: 52%|█████▏ | 1.28GB / 2.49GB  Processing Files (0 / 9) : 66%|██████▌ | 14.6GB / 22.2GB, 955MB/s New Data Upload : 76%|███████▌ | 3.69GB / 4.88GB, 218MB/s  ...nts/optim_095000_rank1.pt: 74%|███████▍ | 1.84GB / 2.49GB  ...nts/optim_095000_rank3.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_095000_rank4.pt: 73%|███████▎ | 1.81GB / 2.49GB  ...nts/optim_095000_rank5.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_095000_rank6.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_095000_rank7.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_095000_rank0.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...eckpoints/model_095000.pt: 22%|██▏ | 512MB / 2.35GB  ...nts/optim_095000_rank2.pt: 56%|█████▌ | 1.39GB / 2.49GB  Processing Files (0 / 9) : 67%|██████▋ | 14.9GB / 22.2GB, 963MB/s New Data Upload : 75%|███████▌ | 3.73GB / 4.94GB, 219MB/s  ...nts/optim_095000_rank1.pt: 75%|███████▍ | 1.86GB / 2.49GB  ...nts/optim_095000_rank3.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_095000_rank4.pt: 74%|███████▎ | 1.83GB / 2.49GB  ...nts/optim_095000_rank5.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_095000_rank6.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_095000_rank7.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_095000_rank0.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...eckpoints/model_095000.pt: 22%|██▏ | 521MB / 2.35GB  ...nts/optim_095000_rank2.pt: 61%|██████ | 1.51GB / 2.49GB  Processing Files (0 / 9) : 68%|██████▊ | 15.2GB / 22.2GB, 973MB/s New Data Upload : 76%|███████▋ | 3.78GB / 4.94GB, 219MB/s  ...nts/optim_095000_rank1.pt: 76%|███████▌ | 1.88GB / 2.49GB  ...nts/optim_095000_rank3.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_095000_rank4.pt: 74%|███████▍ | 1.85GB / 2.49GB  ...nts/optim_095000_rank5.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_095000_rank6.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_095000_rank7.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_095000_rank0.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...eckpoints/model_095000.pt: 23%|██▎ | 529MB / 2.35GB  ...nts/optim_095000_rank2.pt: 65%|██████▌ | 1.63GB / 2.49GB  Processing Files (0 / 9) : 69%|██████▉ | 15.4GB / 22.2GB, 983MB/s New Data Upload : 76%|███████▌ | 3.82GB / 5.01GB, 220MB/s  ...nts/optim_095000_rank1.pt: 76%|███████▋ | 1.90GB / 2.49GB  ...nts/optim_095000_rank3.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_095000_rank4.pt: 75%|███████▌ | 1.87GB / 2.49GB  ...nts/optim_095000_rank5.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_095000_rank6.pt: 78%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_095000_rank7.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_095000_rank0.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...eckpoints/model_095000.pt: 23%|██▎ | 542MB / 2.35GB  ...nts/optim_095000_rank2.pt: 66%|██████▋ | 1.65GB / 2.49GB  Processing Files (0 / 9) : 70%|███████ | 15.6GB / 22.2GB, 987MB/s New Data Upload : 76%|███████▌ | 3.86GB / 5.08GB, 221MB/s  ...nts/optim_095000_rank1.pt: 77%|███████▋ | 1.92GB / 2.49GB  ...nts/optim_095000_rank3.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_095000_rank4.pt: 76%|███████▌ | 1.89GB / 2.49GB  ...nts/optim_095000_rank5.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_095000_rank6.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_095000_rank7.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_095000_rank0.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...eckpoints/model_095000.pt: 24%|██▎ | 553MB / 2.35GB  ...nts/optim_095000_rank2.pt: 71%|███████ | 1.76GB / 2.49GB  Processing Files (0 / 9) : 71%|███████▏ | 15.9GB / 22.2GB, 1.00GB/s New Data Upload : 77%|███████▋ | 3.91GB / 5.08GB, 223MB/s  ...nts/optim_095000_rank1.pt: 78%|███████▊ | 1.94GB / 2.49GB  ...nts/optim_095000_rank3.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_095000_rank4.pt: 77%|███████▋ | 1.91GB / 2.49GB  ...nts/optim_095000_rank5.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_095000_rank6.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_095000_rank7.pt: 79%|███████▊ | 1.96GB / 2.49GB  ...nts/optim_095000_rank0.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...eckpoints/model_095000.pt: 24%|██▍ | 568MB / 2.35GB  ...nts/optim_095000_rank2.pt: 74%|███████▍ | 1.84GB / 2.49GB  Processing Files (0 / 9) : 72%|███████▏ | 16.1GB / 22.2GB, 1.01GB/s New Data Upload : 77%|███████▋ | 3.96GB / 5.15GB, 225MB/s  ...nts/optim_095000_rank1.pt: 79%|███████▉ | 1.96GB / 2.49GB  ...nts/optim_095000_rank3.pt: 80%|████████ | 1.99GB / 2.49GB  ...nts/optim_095000_rank4.pt: 78%|███████▊ | 1.93GB / 2.49GB  ...nts/optim_095000_rank5.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_095000_rank6.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_095000_rank7.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_095000_rank0.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...eckpoints/model_095000.pt: 25%|██▍ | 582MB / 2.35GB  ...nts/optim_095000_rank2.pt: 77%|███████▋ | 1.91GB / 2.49GB  Processing Files (0 / 9) : 73%|███████▎ | 16.3GB / 22.2GB, 1.02GB/s New Data Upload : 77%|███████▋ | 4.00GB / 5.21GB, 226MB/s  ...nts/optim_095000_rank1.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_095000_rank3.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_095000_rank4.pt: 79%|███████▊ | 1.95GB / 2.49GB  ...nts/optim_095000_rank5.pt: 80%|███████▉ | 1.99GB / 2.49GB  ...nts/optim_095000_rank6.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_095000_rank7.pt: 80%|███████▉ | 1.99GB / 2.49GB  ...nts/optim_095000_rank0.pt: 81%|████████ | 2.00GB / 2.49GB  ...eckpoints/model_095000.pt: 26%|██▌ | 603MB / 2.35GB  ...nts/optim_095000_rank2.pt: 80%|███████▉ | 1.98GB / 2.49GB  Processing Files (0 / 9) : 74%|███████▍ | 16.5GB / 22.2GB, 1.03GB/s New Data Upload : 77%|███████▋ | 4.06GB / 5.28GB, 227MB/s  ...nts/optim_095000_rank1.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_095000_rank3.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_095000_rank4.pt: 79%|███████▉ | 1.97GB / 2.49GB  ...nts/optim_095000_rank5.pt: 81%|████████ | 2.00GB / 2.49GB  ...nts/optim_095000_rank6.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_095000_rank7.pt: 81%|████████ | 2.00GB / 2.49GB  ...nts/optim_095000_rank0.pt: 81%|████████ | 2.02GB / 2.49GB  ...eckpoints/model_095000.pt: 27%|██▋ | 627MB / 2.35GB  ...nts/optim_095000_rank2.pt: 82%|████████▏ | 2.03GB / 2.49GB  Processing Files (0 / 9) : 75%|███████▌ | 16.7GB / 22.2GB, 1.04GB/s New Data Upload : 78%|███████▊ | 4.11GB / 5.28GB, 229MB/s  ...nts/optim_095000_rank1.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_095000_rank3.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_095000_rank4.pt: 80%|███████▉ | 1.98GB / 2.49GB  ...nts/optim_095000_rank5.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_095000_rank6.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_095000_rank7.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_095000_rank0.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...eckpoints/model_095000.pt: 28%|██▊ | 646MB / 2.35GB  ...nts/optim_095000_rank2.pt: 82%|████████▏ | 2.05GB / 2.49GB  Processing Files (0 / 9) : 76%|███████▌ | 16.8GB / 22.2GB, 1.03GB/s New Data Upload : 78%|███████▊ | 4.15GB / 5.35GB, 229MB/s  ...nts/optim_095000_rank1.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_095000_rank3.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_095000_rank4.pt: 80%|████████ | 2.00GB / 2.49GB  ...nts/optim_095000_rank5.pt: 81%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_095000_rank6.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_095000_rank7.pt: 81%|████████▏ | 2.02GB / 2.49GB  ...nts/optim_095000_rank0.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...eckpoints/model_095000.pt: 29%|██▊ | 674MB / 2.35GB  ...nts/optim_095000_rank2.pt: 83%|████████▎ | 2.06GB / 2.49GB  Processing Files (0 / 9) : 76%|███████▋ | 17.0GB / 22.2GB, 1.04GB/s New Data Upload : 79%|███████▊ | 4.20GB / 5.35GB, 231MB/s  ...nts/optim_095000_rank1.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_095000_rank3.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_095000_rank4.pt: 81%|████████ | 2.01GB / 2.49GB  ...nts/optim_095000_rank5.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_095000_rank6.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_095000_rank7.pt: 82%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_095000_rank0.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...eckpoints/model_095000.pt: 29%|██▉ | 691MB / 2.35GB  ...nts/optim_095000_rank2.pt: 83%|████████▎ | 2.07GB / 2.49GB  Processing Files (0 / 9) : 77%|███████▋ | 17.1GB / 22.2GB, 1.04GB/s New Data Upload : 78%|███████▊ | 4.23GB / 5.41GB, 230MB/s  ...nts/optim_095000_rank1.pt: 83%|████████▎ | 2.05GB / 2.49GB  ...nts/optim_095000_rank3.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_095000_rank4.pt: 81%|████████▏ | 2.03GB / 2.49GB  ...nts/optim_095000_rank5.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_095000_rank6.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_095000_rank7.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_095000_rank0.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...eckpoints/model_095000.pt: 31%|███ | 723MB / 2.35GB  ...nts/optim_095000_rank2.pt: 84%|████████▍ | 2.09GB / 2.49GB  Processing Files (0 / 9) : 78%|███████▊ | 17.2GB / 22.2GB, 1.04GB/s New Data Upload : 79%|███████▉ | 4.29GB / 5.41GB, 232MB/s  ...nts/optim_095000_rank1.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_095000_rank3.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_095000_rank4.pt: 82%|████████▏ | 2.04GB / 2.49GB  ...nts/optim_095000_rank5.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_095000_rank6.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_095000_rank7.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_095000_rank0.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...eckpoints/model_095000.pt: 32%|███▏ | 744MB / 2.35GB  ...nts/optim_095000_rank2.pt: 84%|████████▍ | 2.10GB / 2.49GB  Processing Files (0 / 9) : 78%|███████▊ | 17.3GB / 22.2GB, 1.04GB/s New Data Upload : 79%|███████▉ | 4.32GB / 5.48GB, 231MB/s  ...nts/optim_095000_rank1.pt: 84%|████████▎ | 2.08GB / 2.49GB  ...nts/optim_095000_rank3.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_095000_rank4.pt: 82%|████████▏ | 2.05GB / 2.49GB  ...nts/optim_095000_rank5.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_095000_rank6.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_095000_rank7.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_095000_rank0.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...eckpoints/model_095000.pt: 33%|███▎ | 771MB / 2.35GB  ...nts/optim_095000_rank2.pt: 85%|████████▍ | 2.11GB / 2.49GB  Processing Files (0 / 9) : 79%|███████▊ | 17.5GB / 22.2GB, 1.04GB/s New Data Upload : 79%|███████▊ | 4.36GB / 5.55GB, 232MB/s  ...nts/optim_095000_rank1.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_095000_rank3.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_095000_rank4.pt: 83%|████████▎ | 2.06GB / 2.49GB  ...nts/optim_095000_rank5.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_095000_rank6.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_095000_rank7.pt: 84%|████████▍ | 2.08GB / 2.49GB  ...nts/optim_095000_rank0.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...eckpoints/model_095000.pt: 34%|███▍ | 796MB / 2.35GB  ...nts/optim_095000_rank2.pt: 85%|████████▌ | 2.12GB / 2.49GB  Processing Files (0 / 9) : 79%|███████▉ | 17.6GB / 22.2GB, 1.03GB/s New Data Upload : 79%|███████▉ | 4.41GB / 5.55GB, 233MB/s  ...nts/optim_095000_rank1.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_095000_rank3.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_095000_rank4.pt: 83%|████████▎ | 2.07GB / 2.49GB  ...nts/optim_095000_rank5.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_095000_rank6.pt: 86%|████████▋ | 2.14GB / 2.49GB  ...nts/optim_095000_rank7.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_095000_rank0.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...eckpoints/model_095000.pt: 35%|███▍ | 814MB / 2.35GB  ...nts/optim_095000_rank2.pt: 86%|████████▌ | 2.13GB / 2.49GB  Processing Files (0 / 9) : 80%|███████▉ | 17.7GB / 22.2GB, 1.02GB/s New Data Upload : 79%|███████▉ | 4.44GB / 5.61GB, 231MB/s  ...nts/optim_095000_rank1.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_095000_rank3.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_095000_rank4.pt: 84%|████████▍ | 2.09GB / 2.49GB  ...nts/optim_095000_rank5.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_095000_rank6.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_095000_rank7.pt: 85%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_095000_rank0.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...eckpoints/model_095000.pt: 36%|███▌ | 843MB / 2.35GB  ...nts/optim_095000_rank2.pt: 86%|████████▋ | 2.14GB / 2.49GB  Processing Files (0 / 9) : 80%|████████ | 17.8GB / 22.2GB, 1.02GB/s New Data Upload : 80%|███████▉ | 4.48GB / 5.61GB, 231MB/s  ...nts/optim_095000_rank1.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_095000_rank3.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_095000_rank4.pt: 84%|████████▍ | 2.10GB / 2.49GB  ...nts/optim_095000_rank5.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_095000_rank6.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_095000_rank7.pt: 85%|████████▌ | 2.11GB / 2.49GB  ...nts/optim_095000_rank0.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...eckpoints/model_095000.pt: 37%|███▋ | 861MB / 2.35GB  ...nts/optim_095000_rank2.pt: 87%|████████▋ | 2.16GB / 2.49GB  Processing Files (0 / 9) : 81%|████████ | 17.9GB / 22.2GB, 1.01GB/s New Data Upload : 79%|███████▉ | 4.52GB / 5.68GB, 230MB/s  ...nts/optim_095000_rank1.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_095000_rank3.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_095000_rank4.pt: 85%|████████▍ | 2.11GB / 2.49GB  ...nts/optim_095000_rank5.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_095000_rank6.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_095000_rank7.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_095000_rank0.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...eckpoints/model_095000.pt: 38%|███▊ | 884MB / 2.35GB  ...nts/optim_095000_rank2.pt: 87%|████████▋ | 2.16GB / 2.49GB  Processing Files (0 / 9) : 81%|████████ | 18.0GB / 22.2GB, 1.00GB/s New Data Upload : 80%|████████ | 4.55GB / 5.68GB, 229MB/s  ...nts/optim_095000_rank1.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_095000_rank3.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_095000_rank4.pt: 85%|████████▌ | 2.12GB / 2.49GB  ...nts/optim_095000_rank5.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_095000_rank6.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_095000_rank7.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_095000_rank0.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...eckpoints/model_095000.pt: 38%|███▊ | 902MB / 2.35GB  ...nts/optim_095000_rank2.pt: 87%|████████▋ | 2.17GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.1GB / 22.2GB, 993MB/s New Data Upload : 80%|███████▉ | 4.58GB / 5.75GB, 228MB/s  ...nts/optim_095000_rank1.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_095000_rank3.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_095000_rank4.pt: 86%|████████▌ | 2.13GB / 2.49GB  ...nts/optim_095000_rank5.pt: 86%|████████▋ | 2.14GB / 2.49GB  ...nts/optim_095000_rank6.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_095000_rank7.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_095000_rank0.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...eckpoints/model_095000.pt: 39%|███▉ | 920MB / 2.35GB  ...nts/optim_095000_rank2.pt: 88%|████████▊ | 2.18GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.2GB / 22.2GB, 979MB/s New Data Upload : 80%|████████ | 4.61GB / 5.75GB, 226MB/s  ...nts/optim_095000_rank1.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_095000_rank3.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_095000_rank4.pt: 86%|████████▌ | 2.14GB / 2.49GB  ...nts/optim_095000_rank5.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_095000_rank6.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_095000_rank7.pt: 87%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_095000_rank0.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...eckpoints/model_095000.pt: 40%|███▉ | 935MB / 2.35GB  ...nts/optim_095000_rank2.pt: 88%|████████▊ | 2.19GB / 2.49GB  Processing Files (0 / 9) : 82%|████████▏ | 18.3GB / 22.2GB, 970MB/s New Data Upload : 80%|███████▉ | 4.64GB / 5.82GB, 225MB/s  ...nts/optim_095000_rank1.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_095000_rank3.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_095000_rank4.pt: 86%|████████▋ | 2.15GB / 2.49GB  ...nts/optim_095000_rank5.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_095000_rank6.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_095000_rank7.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_095000_rank0.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...eckpoints/model_095000.pt: 41%|████ | 957MB / 2.35GB  ...nts/optim_095000_rank2.pt: 88%|████████▊ | 2.20GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.4GB / 22.2GB, 958MB/s New Data Upload : 80%|████████ | 4.67GB / 5.82GB, 223MB/s  ...nts/optim_095000_rank1.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_095000_rank3.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_095000_rank4.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_095000_rank5.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_095000_rank6.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_095000_rank7.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_095000_rank0.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...eckpoints/model_095000.pt: 41%|████▏ | 973MB / 2.35GB  ...nts/optim_095000_rank2.pt: 89%|████████▉ | 2.21GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.5GB / 22.2GB, 954MB/s New Data Upload : 81%|████████ | 4.70GB / 5.82GB, 222MB/s  ...nts/optim_095000_rank1.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_095000_rank3.pt: 89%|████████▊ | 2.21GB / 2.49GB  ...nts/optim_095000_rank4.pt: 87%|████████▋ | 2.16GB / 2.49GB  ...nts/optim_095000_rank5.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_095000_rank6.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_095000_rank7.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_095000_rank0.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...eckpoints/model_095000.pt: 42%|████▏ | 989MB / 2.35GB  ...nts/optim_095000_rank2.pt: 89%|████████▉ | 2.21GB / 2.49GB  Processing Files (0 / 9) : 83%|████████▎ | 18.6GB / 22.2GB, 943MB/s New Data Upload : 80%|████████ | 4.73GB / 5.88GB, 220MB/s  ...nts/optim_095000_rank1.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_095000_rank3.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_095000_rank4.pt: 87%|████████▋ | 2.17GB / 2.49GB  ...nts/optim_095000_rank5.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_095000_rank6.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_095000_rank7.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_095000_rank0.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...eckpoints/model_095000.pt: 43%|████▎ | 1.01GB / 2.35GB  ...nts/optim_095000_rank2.pt: 89%|████████▉ | 2.22GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▍ | 18.6GB / 22.2GB, 937MB/s New Data Upload : 80%|████████ | 4.76GB / 5.95GB, 219MB/s  ...nts/optim_095000_rank1.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_095000_rank3.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_095000_rank4.pt: 88%|████████▊ | 2.18GB / 2.49GB  ...nts/optim_095000_rank5.pt: 88%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_095000_rank6.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_095000_rank7.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_095000_rank0.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...eckpoints/model_095000.pt: 44%|████▍ | 1.03GB / 2.35GB  ...nts/optim_095000_rank2.pt: 90%|████████▉ | 2.23GB / 2.49GB  Processing Files (0 / 9) : 84%|████████▍ | 18.7GB / 22.2GB, 930MB/s New Data Upload : 80%|███████▉ | 4.80GB / 6.02GB, 218MB/s  ...nts/optim_095000_rank1.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_095000_rank3.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_095000_rank4.pt: 88%|████████▊ | 2.19GB / 2.49GB  ...nts/optim_095000_rank5.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_095000_rank6.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_095000_rank7.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_095000_rank0.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...eckpoints/model_095000.pt: 45%|████▍ | 1.06GB / 2.35GB  ...nts/optim_095000_rank2.pt: 90%|█████████ | 2.24GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▍ | 18.8GB / 22.2GB, 919MB/s New Data Upload : 80%|████████ | 4.83GB / 6.02GB, 216MB/s  ...nts/optim_095000_rank1.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_095000_rank3.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_095000_rank4.pt: 89%|████████▊ | 2.20GB / 2.49GB  ...nts/optim_095000_rank5.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_095000_rank6.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_095000_rank7.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_095000_rank0.pt: 90%|█████████ | 2.25GB / 2.49GB  ...eckpoints/model_095000.pt: 46%|████▌ | 1.08GB / 2.35GB  ...nts/optim_095000_rank2.pt: 90%|█████████ | 2.25GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▌ | 18.9GB / 22.2GB, 914MB/s New Data Upload : 81%|████████ | 4.87GB / 6.02GB, 215MB/s  ...nts/optim_095000_rank1.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_095000_rank3.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_095000_rank4.pt: 89%|████████▉ | 2.21GB / 2.49GB  ...nts/optim_095000_rank5.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_095000_rank6.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_095000_rank7.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_095000_rank0.pt: 91%|█████████ | 2.25GB / 2.49GB  ...eckpoints/model_095000.pt: 47%|████▋ | 1.10GB / 2.35GB  ...nts/optim_095000_rank2.pt: 91%|█████████ | 2.25GB / 2.49GB  Processing Files (0 / 9) : 85%|████████▌ | 19.0GB / 22.2GB, 907MB/s New Data Upload : 80%|████████ | 4.90GB / 6.08GB, 214MB/s  ...nts/optim_095000_rank1.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_095000_rank3.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_095000_rank4.pt: 89%|████████▉ | 2.22GB / 2.49GB  ...nts/optim_095000_rank5.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_095000_rank6.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_095000_rank7.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_095000_rank0.pt: 91%|█████████ | 2.26GB / 2.49GB  ...eckpoints/model_095000.pt: 48%|████▊ | 1.12GB / 2.35GB  ...nts/optim_095000_rank2.pt: 91%|█████████ | 2.26GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▌ | 19.1GB / 22.2GB, 901MB/s New Data Upload : 81%|████████ | 4.92GB / 6.08GB, 213MB/s  ...nts/optim_095000_rank1.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_095000_rank3.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_095000_rank4.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_095000_rank5.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_095000_rank6.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_095000_rank7.pt: 90%|████████▉ | 2.24GB / 2.49GB  ...nts/optim_095000_rank0.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...eckpoints/model_095000.pt: 49%|████▊ | 1.14GB / 2.35GB  ...nts/optim_095000_rank2.pt: 91%|█████████▏| 2.27GB / 2.49GB  Processing Files (0 / 9) : 86%|████████▌ | 19.2GB / 22.2GB, 894MB/s New Data Upload : 82%|████████▏ | 4.96GB / 6.08GB, 212MB/s  ...nts/optim_095000_rank1.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_095000_rank3.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_095000_rank4.pt: 90%|████████▉ | 2.23GB / 2.49GB  ...nts/optim_095000_rank5.pt: 90%|█████████ | 2.25GB / 2.49GB  ...nts/optim_095000_rank6.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_095000_rank7.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_095000_rank0.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...eckpoints/model_095000.pt: 50%|████▉ | 1.17GB / 2.35GB  ...nts/optim_095000_rank2.pt: 92%|█████████▏| 2.28GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.3GB / 22.2GB, 886MB/s New Data Upload : 81%|████████▏ | 5.00GB / 6.15GB, 210MB/s  ...nts/optim_095000_rank1.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_095000_rank3.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_095000_rank4.pt: 90%|█████████ | 2.24GB / 2.49GB  ...nts/optim_095000_rank5.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_095000_rank6.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_095000_rank7.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_095000_rank0.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...eckpoints/model_095000.pt: 51%|█████ | 1.19GB / 2.35GB  ...nts/optim_095000_rank2.pt: 92%|█████████▏| 2.28GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.3GB / 22.2GB, 878MB/s New Data Upload : 82%|████████▏ | 5.04GB / 6.15GB, 209MB/s  ...nts/optim_095000_rank1.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_095000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_095000_rank4.pt: 91%|█████████ | 2.25GB / 2.49GB  ...nts/optim_095000_rank5.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_095000_rank6.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_095000_rank7.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_095000_rank0.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...eckpoints/model_095000.pt: 52%|█████▏ | 1.21GB / 2.35GB  ...nts/optim_095000_rank2.pt: 92%|█████████▏| 2.29GB / 2.49GB  Processing Files (0 / 9) : 87%|████████▋ | 19.4GB / 22.2GB, 868MB/s New Data Upload : 81%|████████▏ | 5.06GB / 6.22GB, 207MB/s  ...nts/optim_095000_rank1.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_095000_rank3.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_095000_rank4.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_095000_rank5.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_095000_rank6.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_095000_rank7.pt: 91%|█████████ | 2.26GB / 2.49GB  ...nts/optim_095000_rank0.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...eckpoints/model_095000.pt: 53%|█████▎ | 1.24GB / 2.35GB  ...nts/optim_095000_rank2.pt: 92%|█████████▏| 2.30GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.5GB / 22.2GB, 855MB/s New Data Upload : 81%|████████ | 5.10GB / 6.29GB, 206MB/s  ...nts/optim_095000_rank1.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_095000_rank3.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_095000_rank4.pt: 91%|█████████ | 2.27GB / 2.49GB  ...nts/optim_095000_rank5.pt: 92%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_095000_rank6.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_095000_rank7.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_095000_rank0.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...eckpoints/model_095000.pt: 54%|█████▍ | 1.26GB / 2.35GB  ...nts/optim_095000_rank2.pt: 93%|█████████▎| 2.30GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.6GB / 22.2GB, 845MB/s New Data Upload : 82%|████████▏ | 5.13GB / 6.29GB, 204MB/s  ...nts/optim_095000_rank1.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_095000_rank3.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_095000_rank4.pt: 91%|█████████▏| 2.27GB / 2.49GB  ...nts/optim_095000_rank5.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_095000_rank6.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_095000_rank7.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_095000_rank0.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...eckpoints/model_095000.pt: 55%|█████▌ | 1.30GB / 2.35GB  ...nts/optim_095000_rank2.pt: 93%|█████████▎| 2.31GB / 2.49GB  Processing Files (0 / 9) : 88%|████████▊ | 19.6GB / 22.2GB, 832MB/s New Data Upload : 81%|████████▏ | 5.17GB / 6.35GB, 203MB/s  ...nts/optim_095000_rank1.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_095000_rank3.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_095000_rank4.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_095000_rank5.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_095000_rank6.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_095000_rank7.pt: 92%|█████████▏| 2.28GB / 2.49GB  ...nts/optim_095000_rank0.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...eckpoints/model_095000.pt: 56%|█████▋ | 1.32GB / 2.35GB  ...nts/optim_095000_rank2.pt: 93%|█████████▎| 2.31GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▊ | 19.7GB / 22.2GB, 808MB/s New Data Upload : 81%|████████ | 5.21GB / 6.42GB, 199MB/s  ...nts/optim_095000_rank1.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_095000_rank3.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_095000_rank4.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_095000_rank5.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_095000_rank6.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_095000_rank7.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_095000_rank0.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...eckpoints/model_095000.pt: 58%|█████▊ | 1.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 93%|█████████▎| 2.32GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▉ | 19.8GB / 22.2GB, 790MB/s New Data Upload : 82%|████████▏ | 5.25GB / 6.42GB, 199MB/s  ...nts/optim_095000_rank1.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_095000_rank3.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_095000_rank4.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_095000_rank5.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_095000_rank6.pt: 94%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_095000_rank7.pt: 92%|█████████▏| 2.29GB / 2.49GB  ...nts/optim_095000_rank0.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...eckpoints/model_095000.pt: 59%|█████▉ | 1.39GB / 2.35GB  ...nts/optim_095000_rank2.pt: 93%|█████████▎| 2.32GB / 2.49GB  Processing Files (0 / 9) : 89%|████████▉ | 19.9GB / 22.2GB, 762MB/s New Data Upload : 82%|████████▏ | 5.29GB / 6.42GB, 197MB/s  ...nts/optim_095000_rank1.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_095000_rank3.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_095000_rank4.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_095000_rank5.pt: 93%|█████████▎| 2.30GB / 2.49GB  ...nts/optim_095000_rank6.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_095000_rank7.pt: 92%|█████████▏| 2.30GB / 2.49GB  ...nts/optim_095000_rank0.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...eckpoints/model_095000.pt: 60%|██████ | 1.41GB / 2.35GB  ...nts/optim_095000_rank2.pt: 94%|█████████▎| 2.33GB / 2.49GB  Processing Files (0 / 9) : 90%|████████▉ | 19.9GB / 22.2GB, 751MB/s New Data Upload : 82%|████████▏ | 5.32GB / 6.49GB, 197MB/s  ...nts/optim_095000_rank1.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_095000_rank3.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_095000_rank4.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_095000_rank5.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_095000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_095000_rank7.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_095000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...eckpoints/model_095000.pt: 61%|██████ | 1.43GB / 2.35GB  ...nts/optim_095000_rank2.pt: 94%|█████████▍| 2.34GB / 2.49GB  Processing Files (0 / 9) : 90%|█████████ | 20.0GB / 22.2GB, 725MB/s New Data Upload : 83%|████████▎ | 5.36GB / 6.49GB, 195MB/s  ...nts/optim_095000_rank1.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_095000_rank3.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_095000_rank4.pt: 93%|█████████▎| 2.31GB / 2.49GB  ...nts/optim_095000_rank5.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_095000_rank6.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_095000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_095000_rank0.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...eckpoints/model_095000.pt: 63%|██████▎ | 1.47GB / 2.35GB  ...nts/optim_095000_rank2.pt: 94%|█████████▍| 2.34GB / 2.49GB  Processing Files (0 / 9) : 90%|█████████ | 20.1GB / 22.2GB, 709MB/s New Data Upload : 83%|████████▎ | 5.41GB / 6.49GB, 196MB/s  ...nts/optim_095000_rank1.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_095000_rank3.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_095000_rank4.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_095000_rank5.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_095000_rank6.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_095000_rank7.pt: 93%|█████████▎| 2.32GB / 2.49GB  ...nts/optim_095000_rank0.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...eckpoints/model_095000.pt: 64%|██████▎ | 1.50GB / 2.35GB  ...nts/optim_095000_rank2.pt: 95%|█████████▍| 2.35GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.2GB / 22.2GB, 673MB/s New Data Upload : 84%|████████▍ | 5.44GB / 6.49GB, 194MB/s  ...nts/optim_095000_rank1.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_095000_rank3.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_095000_rank4.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_095000_rank5.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_095000_rank6.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_095000_rank7.pt: 94%|█████████▎| 2.33GB / 2.49GB  ...nts/optim_095000_rank0.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...eckpoints/model_095000.pt: 65%|██████▌ | 1.53GB / 2.35GB  ...nts/optim_095000_rank2.pt: 95%|█████████▍| 2.35GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████ | 20.3GB / 22.2GB, 654MB/s New Data Upload : 85%|████████▍ | 5.49GB / 6.49GB, 193MB/s  ...nts/optim_095000_rank1.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_095000_rank3.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_095000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_095000_rank5.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_095000_rank6.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_095000_rank7.pt: 94%|█████████▍| 2.33GB / 2.49GB  ...nts/optim_095000_rank0.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...eckpoints/model_095000.pt: 66%|██████▋ | 1.56GB / 2.35GB  ...nts/optim_095000_rank2.pt: 95%|█████████▍| 2.36GB / 2.49GB  Processing Files (0 / 9) : 91%|█████████▏| 20.3GB / 22.2GB, 639MB/s New Data Upload : 85%|████████▌ | 5.53GB / 6.49GB, 193MB/s  ...nts/optim_095000_rank1.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_095000_rank3.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_095000_rank4.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_095000_rank5.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_095000_rank6.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_095000_rank7.pt: 94%|█████████▍| 2.34GB / 2.49GB  ...nts/optim_095000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...eckpoints/model_095000.pt: 67%|██████▋ | 1.59GB / 2.35GB  ...nts/optim_095000_rank2.pt: 95%|█████████▌| 2.37GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.4GB / 22.2GB, 617MB/s New Data Upload : 86%|████████▌ | 5.56GB / 6.49GB, 193MB/s  ...nts/optim_095000_rank1.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_095000_rank3.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_095000_rank4.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_095000_rank5.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_095000_rank6.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_095000_rank7.pt: 94%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_095000_rank0.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...eckpoints/model_095000.pt: 69%|██████▉ | 1.62GB / 2.35GB  ...nts/optim_095000_rank2.pt: 95%|█████████▌| 2.37GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.5GB / 22.2GB, 601MB/s New Data Upload : 86%|████████▋ | 5.60GB / 6.49GB, 193MB/s  ...nts/optim_095000_rank1.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_095000_rank3.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_095000_rank4.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_095000_rank5.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_095000_rank6.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_095000_rank7.pt: 95%|█████████▍| 2.35GB / 2.49GB  ...nts/optim_095000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...eckpoints/model_095000.pt: 70%|███████ | 1.64GB / 2.35GB  ...nts/optim_095000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (0 / 9) : 92%|█████████▏| 20.6GB / 22.2GB, 581MB/s New Data Upload : 87%|████████▋ | 5.64GB / 6.49GB, 191MB/s  ...nts/optim_095000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_095000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_095000_rank4.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_095000_rank5.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_095000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_095000_rank7.pt: 95%|█████████▍| 2.36GB / 2.49GB  ...nts/optim_095000_rank0.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...eckpoints/model_095000.pt: 71%|███████ | 1.67GB / 2.35GB  ...nts/optim_095000_rank2.pt: 96%|█████████▌| 2.38GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.6GB / 22.2GB, 563MB/s New Data Upload : 87%|████████▋ | 5.67GB / 6.49GB, 191MB/s  ...nts/optim_095000_rank1.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_095000_rank3.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_095000_rank4.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_095000_rank5.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_095000_rank6.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_095000_rank7.pt: 95%|█████████▌| 2.36GB / 2.49GB  ...nts/optim_095000_rank0.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...eckpoints/model_095000.pt: 72%|███████▏ | 1.70GB / 2.35GB  ...nts/optim_095000_rank2.pt: 96%|█████████▌| 2.39GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.7GB / 22.2GB, 542MB/s New Data Upload : 88%|████████▊ | 5.71GB / 6.49GB, 189MB/s  ...nts/optim_095000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_095000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_095000_rank4.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_095000_rank5.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_095000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_095000_rank7.pt: 95%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_095000_rank0.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...eckpoints/model_095000.pt: 73%|███████▎ | 1.72GB / 2.35GB  ...nts/optim_095000_rank2.pt: 96%|█████████▌| 2.39GB / 2.49GB  Processing Files (0 / 9) : 93%|█████████▎| 20.8GB / 22.2GB, 524MB/s New Data Upload : 88%|████████▊ | 5.74GB / 6.49GB, 188MB/s  ...nts/optim_095000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_095000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_095000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_095000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_095000_rank6.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_095000_rank7.pt: 96%|█████████▌| 2.37GB / 2.49GB  ...nts/optim_095000_rank0.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...eckpoints/model_095000.pt: 74%|███████▍ | 1.75GB / 2.35GB  ...nts/optim_095000_rank2.pt: 96%|█████████▋| 2.39GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▎| 20.8GB / 22.2GB, 512MB/s New Data Upload : 89%|████████▉ | 5.77GB / 6.49GB, 187MB/s  ...nts/optim_095000_rank1.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_095000_rank3.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_095000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_095000_rank5.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_095000_rank6.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_095000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_095000_rank0.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...eckpoints/model_095000.pt: 75%|███████▌ | 1.77GB / 2.35GB  ...nts/optim_095000_rank2.pt: 96%|█████████▋| 2.40GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 20.9GB / 22.2GB, 490MB/s New Data Upload : 89%|████████▉ | 5.79GB / 6.49GB, 185MB/s  ...nts/optim_095000_rank1.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_095000_rank3.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_095000_rank4.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_095000_rank5.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_095000_rank6.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_095000_rank7.pt: 96%|█████████▌| 2.38GB / 2.49GB  ...nts/optim_095000_rank0.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...eckpoints/model_095000.pt: 76%|███████▋ | 1.79GB / 2.35GB  ...nts/optim_095000_rank2.pt: 97%|█████████▋| 2.40GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 20.9GB / 22.2GB, 473MB/s New Data Upload : 90%|████████▉ | 5.82GB / 6.49GB, 183MB/s  ...nts/optim_095000_rank1.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_095000_rank3.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_095000_rank4.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_095000_rank5.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_095000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_095000_rank7.pt: 96%|█████████▌| 2.39GB / 2.49GB  ...nts/optim_095000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...eckpoints/model_095000.pt: 77%|███████▋ | 1.82GB / 2.35GB  ...nts/optim_095000_rank2.pt: 97%|█████████▋| 2.40GB / 2.49GB  Processing Files (0 / 9) : 94%|█████████▍| 21.0GB / 22.2GB, 459MB/s New Data Upload : 90%|█████████ | 5.86GB / 6.49GB, 181MB/s  ...nts/optim_095000_rank1.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_095000_rank3.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_095000_rank4.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_095000_rank5.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_095000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_095000_rank7.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_095000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...eckpoints/model_095000.pt: 78%|███████▊ | 1.84GB / 2.35GB  ...nts/optim_095000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▍| 21.0GB / 22.2GB, 441MB/s New Data Upload : 91%|█████████ | 5.88GB / 6.49GB, 179MB/s  ...nts/optim_095000_rank1.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_095000_rank3.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_095000_rank4.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_095000_rank5.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_095000_rank6.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_095000_rank7.pt: 96%|█████████▋| 2.39GB / 2.49GB  ...nts/optim_095000_rank0.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...eckpoints/model_095000.pt: 80%|███████▉ | 1.87GB / 2.35GB  ...nts/optim_095000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▍| 21.1GB / 22.2GB, 427MB/s New Data Upload : 91%|█████████ | 5.91GB / 6.49GB, 177MB/s  ...nts/optim_095000_rank1.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_095000_rank3.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_095000_rank4.pt: 96%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_095000_rank5.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_095000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_095000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...eckpoints/model_095000.pt: 81%|████████ | 1.89GB / 2.35GB  ...nts/optim_095000_rank2.pt: 97%|█████████▋| 2.41GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.1GB / 22.2GB, 420MB/s New Data Upload : 92%|█████████▏| 5.94GB / 6.49GB, 176MB/s  ...nts/optim_095000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank4.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_095000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank7.pt: 97%|█████████▋| 2.40GB / 2.49GB  ...nts/optim_095000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...eckpoints/model_095000.pt: 82%|████████▏ | 1.92GB / 2.35GB  ...nts/optim_095000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.2GB / 22.2GB, 412MB/s New Data Upload : 92%|█████████▏| 5.97GB / 6.49GB, 173MB/s  ...nts/optim_095000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank6.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank0.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...eckpoints/model_095000.pt: 82%|████████▏ | 1.94GB / 2.35GB  ...nts/optim_095000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (0 / 9) : 95%|█████████▌| 21.2GB / 22.2GB, 406MB/s New Data Upload : 92%|█████████▏| 5.99GB / 6.49GB, 173MB/s  ...nts/optim_095000_rank1.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_095000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...eckpoints/model_095000.pt: 83%|████████▎ | 1.95GB / 2.35GB  ...nts/optim_095000_rank2.pt: 97%|█████████▋| 2.42GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 394MB/s New Data Upload : 93%|█████████▎| 6.01GB / 6.49GB, 169MB/s  ...nts/optim_095000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_095000_rank3.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank4.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank5.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank6.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_095000_rank7.pt: 97%|█████████▋| 2.41GB / 2.49GB  ...nts/optim_095000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...eckpoints/model_095000.pt: 84%|████████▍ | 1.98GB / 2.35GB  ...nts/optim_095000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.3GB / 22.2GB, 392MB/s New Data Upload : 93%|█████████▎| 6.04GB / 6.49GB, 169MB/s  ...nts/optim_095000_rank1.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_095000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_095000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_095000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_095000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_095000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_095000_rank0.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...eckpoints/model_095000.pt: 85%|████████▌ | 2.00GB / 2.35GB  ...nts/optim_095000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▌| 21.4GB / 22.2GB, 384MB/s New Data Upload : 94%|█████████▎| 6.07GB / 6.49GB, 167MB/s  ...nts/optim_095000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_095000_rank3.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_095000_rank4.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_095000_rank5.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_095000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_095000_rank7.pt: 97%|█████████▋| 2.42GB / 2.49GB  ...nts/optim_095000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...eckpoints/model_095000.pt: 86%|████████▌ | 2.02GB / 2.35GB  ...nts/optim_095000_rank2.pt: 98%|█████████▊| 2.43GB / 2.49GB  Processing Files (0 / 9) : 96%|█████████▋| 21.4GB / 22.2GB, 375MB/s New Data Upload : 94%|█████████▍| 6.09GB / 6.49GB, 165MB/s  ...nts/optim_095000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_095000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_095000_rank4.pt: 98%|█████████▊| 2.42GB / 2.49GB  ...nts/optim_095000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_095000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_095000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_095000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...eckpoints/model_095000.pt: 87%|████████▋ | 2.04GB / 2.35GB  ...nts/optim_095000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 370MB/s New Data Upload : 94%|█████████▍| 6.11GB / 6.49GB, 164MB/s  ...nts/optim_095000_rank1.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_095000_rank3.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_095000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_095000_rank5.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_095000_rank6.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_095000_rank7.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_095000_rank0.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...eckpoints/model_095000.pt: 87%|████████▋ | 2.05GB / 2.35GB  ...nts/optim_095000_rank2.pt: 98%|█████████▊| 2.44GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.5GB / 22.2GB, 362MB/s New Data Upload : 94%|█████████▍| 6.13GB / 6.49GB, 162MB/s  ...nts/optim_095000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank4.pt: 98%|█████████▊| 2.43GB / 2.49GB  ...nts/optim_095000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank0.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_095000.pt: 88%|████████▊ | 2.07GB / 2.35GB  ...nts/optim_095000_rank2.pt: 98%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 358MB/s New Data Upload : 95%|█████████▍| 6.16GB / 6.49GB, 161MB/s  ...nts/optim_095000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_095000.pt: 89%|████████▉ | 2.09GB / 2.35GB  ...nts/optim_095000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.6GB / 22.2GB, 354MB/s New Data Upload : 95%|█████████▌| 6.18GB / 6.49GB, 160MB/s  ...nts/optim_095000_rank1.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank6.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank0.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...eckpoints/model_095000.pt: 90%|████████▉ | 2.11GB / 2.35GB  ...nts/optim_095000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 97%|█████████▋| 21.7GB / 22.2GB, 347MB/s New Data Upload : 96%|█████████▌| 6.20GB / 6.49GB, 158MB/s  ...nts/optim_095000_rank1.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank3.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank5.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank6.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank7.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank0.pt: 99%|█████████▉| 2.45GB / 2.49GB  ...eckpoints/model_095000.pt: 90%|█████████ | 2.12GB / 2.35GB  ...nts/optim_095000_rank2.pt: 99%|█████████▊| 2.45GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 340MB/s New Data Upload : 96%|█████████▌| 6.22GB / 6.49GB, 157MB/s  ...nts/optim_095000_rank1.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank4.pt: 98%|█████████▊| 2.44GB / 2.49GB  ...nts/optim_095000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank6.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank7.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_095000.pt: 91%|█████████ | 2.13GB / 2.35GB  ...nts/optim_095000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.7GB / 22.2GB, 335MB/s New Data Upload : 96%|█████████▌| 6.23GB / 6.49GB, 156MB/s  ...nts/optim_095000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank3.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank4.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank5.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank7.pt: 98%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_095000.pt: 92%|█████████▏| 2.15GB / 2.35GB  ...nts/optim_095000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 330MB/s New Data Upload : 96%|█████████▋| 6.25GB / 6.49GB, 155MB/s  ...nts/optim_095000_rank1.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_095000.pt: 92%|█████████▏| 2.16GB / 2.35GB  ...nts/optim_095000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 325MB/s New Data Upload : 97%|█████████▋| 6.27GB / 6.49GB, 154MB/s  ...nts/optim_095000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank3.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank5.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank6.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank7.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank0.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...eckpoints/model_095000.pt: 93%|█████████▎| 2.18GB / 2.35GB  ...nts/optim_095000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.8GB / 22.2GB, 321MB/s New Data Upload : 97%|█████████▋| 6.28GB / 6.49GB, 152MB/s  ...nts/optim_095000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank4.pt: 99%|█████████▊| 2.45GB / 2.49GB  ...nts/optim_095000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_095000.pt: 93%|█████████▎| 2.19GB / 2.35GB  ...nts/optim_095000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 315MB/s New Data Upload : 97%|█████████▋| 6.30GB / 6.49GB, 151MB/s  ...nts/optim_095000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_095000.pt: 94%|█████████▍| 2.21GB / 2.35GB  ...nts/optim_095000_rank2.pt: 99%|█████████▉| 2.46GB / 2.49GB  Processing Files (0 / 9) : 98%|█████████▊| 21.9GB / 22.2GB, 309MB/s New Data Upload : 97%|█████████▋| 6.32GB / 6.49GB, 149MB/s  ...nts/optim_095000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_095000.pt: 95%|█████████▍| 2.22GB / 2.35GB  ...nts/optim_095000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 303MB/s New Data Upload : 98%|█████████▊| 6.33GB / 6.49GB, 147MB/s  ...nts/optim_095000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_095000.pt: 95%|█████████▌| 2.23GB / 2.35GB  ...nts/optim_095000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▊| 21.9GB / 22.2GB, 295MB/s New Data Upload : 98%|█████████▊| 6.35GB / 6.49GB, 145MB/s  ...nts/optim_095000_rank1.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank3.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank5.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank6.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank7.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_095000.pt: 96%|█████████▌| 2.24GB / 2.35GB  ...nts/optim_095000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 291MB/s New Data Upload : 98%|█████████▊| 6.36GB / 6.49GB, 144MB/s  ...nts/optim_095000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank4.pt: 99%|█████████▉| 2.46GB / 2.49GB  ...nts/optim_095000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank0.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_095000.pt: 96%|█████████▌| 2.25GB / 2.35GB  ...nts/optim_095000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 287MB/s New Data Upload : 98%|█████████▊| 6.37GB / 6.49GB, 142MB/s  ...nts/optim_095000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...eckpoints/model_095000.pt: 96%|█████████▋| 2.26GB / 2.35GB  ...nts/optim_095000_rank2.pt: 99%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 279MB/s New Data Upload : 98%|█████████▊| 6.38GB / 6.49GB, 139MB/s  ...nts/optim_095000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_095000.pt: 97%|█████████▋| 2.27GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|█████████▉| 2.47GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.0GB / 22.2GB, 273MB/s New Data Upload : 99%|█████████▊| 6.40GB / 6.49GB, 137MB/s  ...nts/optim_095000_rank1.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank3.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank4.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank5.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank6.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank7.pt: 99%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_095000.pt: 97%|█████████▋| 2.28GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 267MB/s New Data Upload : 99%|█████████▉| 6.41GB / 6.49GB, 135MB/s  ...nts/optim_095000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|█████████▉| 2.47GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_095000.pt: 98%|█████████▊| 2.29GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 263MB/s New Data Upload : 99%|█████████▉| 6.42GB / 6.49GB, 133MB/s  ...nts/optim_095000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_095000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 99%|█████████▉| 22.1GB / 22.2GB, 258MB/s New Data Upload : 99%|█████████▉| 6.43GB / 6.49GB, 130MB/s  ...nts/optim_095000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_095000.pt: 98%|█████████▊| 2.30GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.1GB / 22.2GB, 253MB/s New Data Upload : 99%|█████████▉| 6.44GB / 6.49GB, 128MB/s  ...nts/optim_095000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...eckpoints/model_095000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 248MB/s New Data Upload : 99%|█████████▉| 6.44GB / 6.49GB, 124MB/s  ...nts/optim_095000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 242MB/s New Data Upload : 99%|█████████▉| 6.45GB / 6.49GB, 121MB/s  ...nts/optim_095000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 98%|█████████▊| 2.31GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|█████████▉| 2.48GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 234MB/s New Data Upload : 99%|█████████▉| 6.45GB / 6.49GB, 118MB/s  ...nts/optim_095000_rank1.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|█████████▉| 2.48GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 99%|█████████▊| 2.32GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 229MB/s New Data Upload : 99%|█████████▉| 6.46GB / 6.49GB, 114MB/s  ...nts/optim_095000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 223MB/s New Data Upload : 100%|█████████▉| 6.46GB / 6.49GB, 112MB/s  ...nts/optim_095000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 99%|█████████▉| 2.32GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 215MB/s New Data Upload : 100%|█████████▉| 6.47GB / 6.49GB, 108MB/s  ...nts/optim_095000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (0 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 207MB/s New Data Upload : 100%|█████████▉| 6.47GB / 6.49GB, 104MB/s  ...nts/optim_095000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (1 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 200MB/s New Data Upload : 100%|█████████▉| 6.47GB / 6.49GB, 101MB/s  ...nts/optim_095000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 99%|█████████▉| 2.33GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (1 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 193MB/s New Data Upload : 100%|█████████▉| 6.48GB / 6.49GB, 97.0MB/s  ...nts/optim_095000_rank1.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|█████████▉| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|█████████▉| 2.49GB / 2.49GB  Processing Files (1 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 185MB/s New Data Upload : 100%|█████████▉| 6.48GB / 6.49GB, 93.4MB/s  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|█████████▉| 2.34GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 178MB/s New Data Upload : 100%|█████████▉| 6.48GB / 6.49GB, 90.3MB/s  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 170MB/s New Data Upload : 100%|█████████▉| 6.49GB / 6.49GB, 86.8MB/s  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (8 / 9) : 100%|█████████▉| 22.2GB / 22.2GB, 164MB/s New Data Upload : 100%|█████████▉| 6.49GB / 6.49GB, 83.3MB/s  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|█████████▉| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (9 / 9) : 100%|██████████| 22.2GB / 22.2GB, 128MB/s New Data Upload : 100%|██████████| 6.49GB / 6.49GB, 65.5MB/s  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB  ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB  ...eckpoints/model_095000.pt: 100%|██████████| 2.35GB / 2.35GB  ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB  Processing Files (9 / 9) : 100%|██████████| 22.2GB / 22.2GB, 64.9MB/s New Data Upload : 100%|██████████| 6.49GB / 6.49GB, 33.0MB/s ...nts/optim_095000_rank1.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_095000_rank3.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_095000_rank4.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_095000_rank5.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_095000_rank6.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_095000_rank7.pt: 100%|██████████| 2.49GB / 2.49GB ...nts/optim_095000_rank0.pt: 100%|██████████| 2.49GB / 2.49GB ...eckpoints/model_095000.pt: 100%|██████████| 2.35GB / 2.35GB ...nts/optim_095000_rank2.pt: 100%|██████████| 2.49GB / 2.49GB