| [2026-05-05 17:59:14] starting temp=0.96 lambda grid on GPUs: 0,1,2,3,4,5,6,7 |
| [2026-05-05 17:59:14] out_root=/home/sunhc/diffusion/sad/eval/mauve_temp096_lambda_grid_8gpu |
| [2026-05-05 17:59:14] num_samples=1024 sample_batch_size=256 mauve_batch_size=16 |
| [2026-05-05 17:59:14] pps_list=1,2,4 lambda_list=0.2,0.4,0.6,0.8 |
| [2026-05-05 17:59:14] models=top1,top2,top3,h2 |
| [2026-05-05 17:59:14] launched top1_pps1_lam0.2_temp0.96 on gpu0 pid=180965 |
| [2026-05-05 17:59:14] launched top2_pps1_lam0.2_temp0.96 on gpu1 pid=180971 |
| [2026-05-05 17:59:14] launched top3_pps1_lam0.2_temp0.96 on gpu2 pid=180977 |
| [2026-05-05 17:59:14] launched h2_pps1_lam0.2_temp0.96 on gpu3 pid=180983 |
| [2026-05-05 17:59:14] launched top1_pps1_lam0.4_temp0.96 on gpu4 pid=180989 |
| [2026-05-05 17:59:14] launched top2_pps1_lam0.4_temp0.96 on gpu5 pid=180995 |
| [2026-05-05 17:59:14] launched top3_pps1_lam0.4_temp0.96 on gpu6 pid=181001 |
| [2026-05-05 17:59:14] launched h2_pps1_lam0.4_temp0.96 on gpu7 pid=181007 |
| [2026-05-05 18:05:54] nonzero exit (-1) for top1_pps1_lam0.2_temp0.96 on gpu0 pid=180965 but output exists |
| [2026-05-05 18:05:54] launched top1_pps1_lam0.6_temp0.96 on gpu0 pid=190174 |
| [2026-05-05 18:05:54] finished top2_pps1_lam0.2_temp0.96 on gpu1 pid=180971 |
| [2026-05-05 18:05:54] launched top2_pps1_lam0.6_temp0.96 on gpu1 pid=190183 |
| [2026-05-05 18:06:09] nonzero exit (-1) for top2_pps1_lam0.4_temp0.96 on gpu5 pid=180995 but output exists |
| [2026-05-05 18:06:09] launched top3_pps1_lam0.6_temp0.96 on gpu5 pid=190989 |
| [2026-05-05 18:06:24] nonzero exit (-1) for top1_pps1_lam0.4_temp0.96 on gpu4 pid=180989 but output exists |
| [2026-05-05 18:06:24] launched h2_pps1_lam0.6_temp0.96 on gpu4 pid=191470 |
| [2026-05-05 18:06:44] nonzero exit (-1) for top3_pps1_lam0.2_temp0.96 on gpu2 pid=180977 but output exists |
| [2026-05-05 18:06:44] launched top1_pps1_lam0.8_temp0.96 on gpu2 pid=191820 |
| [2026-05-05 18:06:54] nonzero exit (-1) for top3_pps1_lam0.4_temp0.96 on gpu6 pid=181001 but output exists |
| [2026-05-05 18:06:54] launched top2_pps1_lam0.8_temp0.96 on gpu6 pid=192164 |
| [2026-05-05 18:06:59] nonzero exit (-1) for h2_pps1_lam0.2_temp0.96 on gpu3 pid=180983 but output exists |
| [2026-05-05 18:06:59] launched top3_pps1_lam0.8_temp0.96 on gpu3 pid=192312 |
| [2026-05-05 18:08:24] nonzero exit (-1) for h2_pps1_lam0.4_temp0.96 on gpu7 pid=181007 but output exists |
| [2026-05-05 18:08:24] launched h2_pps1_lam0.8_temp0.96 on gpu7 pid=193022 |
| [2026-05-05 18:11:49] nonzero exit (-1) for top2_pps1_lam0.6_temp0.96 on gpu1 pid=190183 but output exists |
| [2026-05-05 18:11:49] launched top1_pps2_lam0.2_temp0.96 on gpu1 pid=197849 |
| [2026-05-05 18:12:09] nonzero exit (-1) for top1_pps1_lam0.6_temp0.96 on gpu0 pid=190174 but output exists |
| [2026-05-05 18:12:09] launched top2_pps2_lam0.2_temp0.96 on gpu0 pid=198199 |
| [2026-05-05 18:12:49] nonzero exit (-1) for top2_pps1_lam0.8_temp0.96 on gpu6 pid=192164 but output exists |
| [2026-05-05 18:12:49] launched top3_pps2_lam0.2_temp0.96 on gpu6 pid=199520 |
| [2026-05-05 18:13:14] nonzero exit (-1) for top1_pps1_lam0.8_temp0.96 on gpu2 pid=191820 but output exists |
| [2026-05-05 18:13:14] launched h2_pps2_lam0.2_temp0.96 on gpu2 pid=200574 |
| [2026-05-05 18:13:29] nonzero exit (-1) for top3_pps1_lam0.6_temp0.96 on gpu5 pid=190989 but output exists |
| [2026-05-05 18:13:29] launched top1_pps2_lam0.4_temp0.96 on gpu5 pid=201621 |
| [2026-05-05 18:14:24] nonzero exit (-1) for top3_pps1_lam0.8_temp0.96 on gpu3 pid=192312 but output exists |
| [2026-05-05 18:14:24] launched top2_pps2_lam0.4_temp0.96 on gpu3 pid=202261 |
| [2026-05-05 18:15:34] nonzero exit (-1) for h2_pps1_lam0.6_temp0.96 on gpu4 pid=191470 but output exists |
| [2026-05-05 18:15:34] launched top3_pps2_lam0.4_temp0.96 on gpu4 pid=205329 |
| [2026-05-05 18:16:29] nonzero exit (-1) for top1_pps2_lam0.2_temp0.96 on gpu1 pid=197849 but output exists |
| [2026-05-05 18:16:29] launched h2_pps2_lam0.4_temp0.96 on gpu1 pid=206794 |
| [2026-05-05 18:16:44] nonzero exit (-1) for top2_pps2_lam0.2_temp0.96 on gpu0 pid=198199 but output exists |
| [2026-05-05 18:16:44] launched top1_pps2_lam0.6_temp0.96 on gpu0 pid=207276 |
| [2026-05-05 18:17:54] nonzero exit (-1) for top3_pps2_lam0.2_temp0.96 on gpu6 pid=199520 but output exists |
| [2026-05-05 18:17:54] launched top2_pps2_lam0.6_temp0.96 on gpu6 pid=208741 |
| [2026-05-05 18:17:59] nonzero exit (-1) for h2_pps1_lam0.8_temp0.96 on gpu7 pid=193022 but output exists |
| [2026-05-05 18:17:59] launched top3_pps2_lam0.6_temp0.96 on gpu7 pid=208887 |
| [2026-05-05 18:18:20] nonzero exit (-1) for top1_pps2_lam0.4_temp0.96 on gpu5 pid=201621 but output exists |
| [2026-05-05 18:18:20] launched h2_pps2_lam0.6_temp0.96 on gpu5 pid=209558 |
| [2026-05-05 18:18:50] nonzero exit (-1) for h2_pps2_lam0.2_temp0.96 on gpu2 pid=200574 but output exists |
| [2026-05-05 18:18:50] launched top1_pps2_lam0.8_temp0.96 on gpu2 pid=210747 |
| [2026-05-05 18:19:05] nonzero exit (-1) for top2_pps2_lam0.4_temp0.96 on gpu3 pid=202261 but output exists |
| [2026-05-05 18:19:05] launched top2_pps2_lam0.8_temp0.96 on gpu3 pid=211095 |
| [2026-05-05 18:20:55] nonzero exit (-1) for top3_pps2_lam0.4_temp0.96 on gpu4 pid=205329 but output exists |
| [2026-05-05 18:20:55] launched top3_pps2_lam0.8_temp0.96 on gpu4 pid=214896 |
| [2026-05-05 18:21:35] nonzero exit (-1) for top1_pps2_lam0.6_temp0.96 on gpu0 pid=207276 but output exists |
| [2026-05-05 18:21:35] launched h2_pps2_lam0.8_temp0.96 on gpu0 pid=216094 |
| [2026-05-05 18:22:30] nonzero exit (-1) for h2_pps2_lam0.4_temp0.96 on gpu1 pid=206794 but output exists |
| [2026-05-05 18:22:30] launched top1_pps4_lam0.2_temp0.96 on gpu1 pid=217114 |
| [2026-05-05 18:22:35] nonzero exit (-1) for top2_pps2_lam0.6_temp0.96 on gpu6 pid=208741 but output exists |
| [2026-05-05 18:22:35] launched top2_pps4_lam0.2_temp0.96 on gpu6 pid=217333 |
| [2026-05-05 18:23:30] nonzero exit (-1) for top3_pps2_lam0.6_temp0.96 on gpu7 pid=208887 but output exists |
| [2026-05-05 18:23:30] launched top3_pps4_lam0.2_temp0.96 on gpu7 pid=219932 |
| [2026-05-05 18:24:00] nonzero exit (-1) for top2_pps2_lam0.8_temp0.96 on gpu3 pid=211095 but output exists |
| [2026-05-05 18:24:00] launched h2_pps4_lam0.2_temp0.96 on gpu3 pid=220289 |
| [2026-05-05 18:24:05] nonzero exit (-1) for top1_pps2_lam0.8_temp0.96 on gpu2 pid=210747 but output exists |
| [2026-05-05 18:24:05] launched top1_pps4_lam0.4_temp0.96 on gpu2 pid=220436 |
| [2026-05-05 18:24:40] nonzero exit (-1) for h2_pps2_lam0.6_temp0.96 on gpu5 pid=209558 but output exists |
| [2026-05-05 18:24:40] launched top2_pps4_lam0.4_temp0.96 on gpu5 pid=222091 |
| [2026-05-05 18:26:50] nonzero exit (-1) for top3_pps2_lam0.8_temp0.96 on gpu4 pid=214896 but output exists |
| [2026-05-05 18:26:50] launched top3_pps4_lam0.4_temp0.96 on gpu4 pid=225851 |
| [2026-05-05 18:27:00] nonzero exit (-1) for top1_pps4_lam0.2_temp0.96 on gpu1 pid=217114 but output exists |
| [2026-05-05 18:27:00] launched h2_pps4_lam0.4_temp0.96 on gpu1 pid=226197 |
| [2026-05-05 18:27:00] finished top2_pps4_lam0.2_temp0.96 on gpu6 pid=217333 |
| [2026-05-05 18:27:00] launched top1_pps4_lam0.6_temp0.96 on gpu6 pid=226206 |
| [2026-05-05 18:27:45] nonzero exit (-1) for top3_pps4_lam0.2_temp0.96 on gpu7 pid=219932 but output exists |
| [2026-05-05 18:27:45] launched top2_pps4_lam0.6_temp0.96 on gpu7 pid=227279 |
| [2026-05-05 18:28:15] nonzero exit (-1) for h2_pps2_lam0.8_temp0.96 on gpu0 pid=216094 but output exists |
| [2026-05-05 18:28:15] launched top3_pps4_lam0.6_temp0.96 on gpu0 pid=228973 |
| [2026-05-05 18:28:25] nonzero exit (-1) for top1_pps4_lam0.4_temp0.96 on gpu2 pid=220436 but output exists |
| [2026-05-05 18:28:25] launched h2_pps4_lam0.6_temp0.96 on gpu2 pid=229383 |
| [2026-05-05 18:28:50] nonzero exit (-1) for h2_pps4_lam0.2_temp0.96 on gpu3 pid=220289 but output exists |
| [2026-05-05 18:28:50] launched top1_pps4_lam0.8_temp0.96 on gpu3 pid=230885 |
| [2026-05-05 18:29:00] nonzero exit (-1) for top2_pps4_lam0.4_temp0.96 on gpu5 pid=222091 but output exists |
| [2026-05-05 18:29:00] launched top2_pps4_lam0.8_temp0.96 on gpu5 pid=231230 |
| [2026-05-05 18:31:35] nonzero exit (-1) for top1_pps4_lam0.6_temp0.96 on gpu6 pid=226206 but output exists |
| [2026-05-05 18:31:35] launched top3_pps4_lam0.8_temp0.96 on gpu6 pid=235536 |
| [2026-05-05 18:31:40] nonzero exit (-1) for top3_pps4_lam0.4_temp0.96 on gpu4 pid=225851 but output exists |
| [2026-05-05 18:31:40] launched h2_pps4_lam0.8_temp0.96 on gpu4 pid=235683 |
| [2026-05-05 18:32:46] finished top3_pps4_lam0.6_temp0.96 on gpu0 pid=228973 |
| [2026-05-05 18:32:46] finished h2_pps4_lam0.4_temp0.96 on gpu1 pid=226197 |
| [2026-05-05 18:33:39] finished h2_pps4_lam0.6_temp0.96 on gpu2 pid=229383 |
| [2026-05-05 18:33:39] finished top1_pps4_lam0.8_temp0.96 on gpu3 pid=230885 |
| [2026-05-05 18:36:36] finished h2_pps4_lam0.8_temp0.96 on gpu4 pid=235683 |
| [2026-05-05 18:36:36] finished top2_pps4_lam0.8_temp0.96 on gpu5 pid=231230 |
| [2026-05-05 18:36:36] finished top3_pps4_lam0.8_temp0.96 on gpu6 pid=235536 |
| [2026-05-05 18:36:36] finished top2_pps4_lam0.6_temp0.96 on gpu7 pid=227279 |
| [2026-05-05 18:36:36] all temp=0.96 lambda grid jobs completed |
|
|
