Add files using upload-large-folder tool

eval/ar_baseline_latest_test/logs/temp0.2_gpu1.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.2)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'The first-ever “Star Wars” movie is set to hit theaters on May 25, 2018, but the film’s director, George Lucas, has alre'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 0.3384644670114785,
+  "median_nll": 0.002919342485256493,
+  "ppl": 1.4027919026796367,
+  "acc": 0.9352297310387137,
+  "tokens": 61942,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.2,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.2_gpu1.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.2_gpu1.json
+Diversity metrics for temp0.2_gpu1.json:
+  distinct_1: 0.0380
+  repetition_1: 0.8770
+  d1: 0.0380
+  r1: 0.8770
+  distinct_2: 0.0906
+  repetition_2: 0.8249
+  d2: 0.0906
+  r2: 0.8249
+  distinct_4: 0.1524
+  repetition_4: 0.7935
+  d4: 0.1524
+  r4: 0.7935
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.2_gpu1.json

eval/ar_baseline_latest_test/logs/temp0.2_gpu3.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.2)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'The U.S. Army Corps of Engineers has approved a plan to build a $1.6 billion, 1,000-mile pipeline that would carry oil f'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 0.35488078943010276,
+  "median_nll": 0.003100768430158496,
+  "ppl": 1.4260106486063449,
+  "acc": 0.9321503131524008,
+  "tokens": 62270,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.2,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.2_gpu3.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.2_gpu3.json
+Diversity metrics for temp0.2_gpu3.json:
+  distinct_1: 0.0375
+  repetition_1: 0.8804
+  d1: 0.0375
+  r1: 0.8804
+  distinct_2: 0.0909
+  repetition_2: 0.8279
+  d2: 0.0909
+  r2: 0.8279
+  distinct_4: 0.1540
+  repetition_4: 0.7965
+  d4: 0.1540
+  r4: 0.7965
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.2_gpu3.json

eval/ar_baseline_latest_test/logs/temp0.2_gpu5.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.2)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'The first time I saw the movie, I was in the middle of a long drive to the airport. I was driving a car, and I was in th'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 0.36794694845510534,
+  "median_nll": 0.0035996540682390332,
+  "ppl": 1.4447653899047765,
+  "acc": 0.9298173021834617,
+  "tokens": 62836,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.2,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.2_gpu5.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.2_gpu5.json
+Diversity metrics for temp0.2_gpu5.json:
+  distinct_1: 0.0388
+  repetition_1: 0.8733
+  d1: 0.0388
+  r1: 0.8733
+  distinct_2: 0.0973
+  repetition_2: 0.8148
+  d2: 0.0973
+  r2: 0.8148
+  distinct_4: 0.1693
+  repetition_4: 0.7799
+  d4: 0.1693
+  r4: 0.7799
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.2_gpu5.json

eval/ar_baseline_latest_test/logs/temp0.4_gpu5.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.4)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'The United States has been on a roll in the global economic and financial crisis, but its efforts to contain the crisis '
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 0.6422196306169079,
+  "median_nll": 0.022978578694164753,
+  "ppl": 1.900695041540618,
+  "acc": 0.8628368331338628,
+  "tokens": 55146,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.4,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.4_gpu5.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.4_gpu5.json
+Diversity metrics for temp0.4_gpu5.json:
+  distinct_1: 0.0767
+  repetition_1: 0.7744
+  d1: 0.0767
+  r1: 0.7744
+  distinct_2: 0.2035
+  repetition_2: 0.6561
+  d2: 0.2035
+  r2: 0.6561
+  distinct_4: 0.3410
+  repetition_4: 0.5816
+  d4: 0.3410
+  r4: 0.5816
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.4_gpu5.json

eval/ar_baseline_latest_test/logs/temp0.4_gpu7.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.4)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'The U.S. Department of Justice has issued a subpoena to the New York Times and its owner, Daniel Richman, seeking docume'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 0.6613957768165036,
+  "median_nll": 0.03448391519486904,
+  "ppl": 1.9374947582386717,
+  "acc": 0.8539113160163424,
+  "tokens": 53358,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.4,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.4_gpu7.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.4_gpu7.json
+Diversity metrics for temp0.4_gpu7.json:
+  distinct_1: 0.0866
+  repetition_1: 0.7545
+  d1: 0.0866
+  r1: 0.7545
+  distinct_2: 0.2257
+  repetition_2: 0.6243
+  d2: 0.2257
+  r2: 0.6243
+  distinct_4: 0.3803
+  repetition_4: 0.5404
+  d4: 0.3803
+  r4: 0.5404
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.4_gpu7.json

eval/ar_baseline_latest_test/logs/temp0.82_gpu6.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.82)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'BETHLEHEM (Ma’an) -- The Palestinian Authority on Wednesday announced it was establishing a new state in the West Bank w'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 2.3033388616360484,
+  "median_nll": 1.5766230821609497,
+  "ppl": 10.007540527969768,
+  "acc": 0.5048616140997784,
+  "tokens": 45561,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.82,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.82_gpu6.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.82_gpu6.json
+Diversity metrics for temp0.82_gpu6.json:
+  distinct_1: 0.2445
+  repetition_1: 0.4226
+  d1: 0.2445
+  r1: 0.4226
+  distinct_2: 0.6910
+  repetition_2: 0.1198
+  d2: 0.6910
+  r2: 0.1198
+  distinct_4: 0.9589
+  repetition_4: 0.0340
+  d4: 0.9589
+  r4: 0.0340
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.82_gpu6.json

eval/ar_baseline_latest_test/logs/temp0.84_gpu0.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.84)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'The protagonist of the Möbius strip is a high school student who is teaching the world about the “Mokai,” the Japanese t'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 2.4762633348149796,
+  "median_nll": 1.748022437095642,
+  "ppl": 11.896727173041405,
+  "acc": 0.477586398488721,
+  "tokens": 44995,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.84,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.84_gpu0.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.84_gpu0.json
+Diversity metrics for temp0.84_gpu0.json:
+  distinct_1: 0.2500
+  repetition_1: 0.4025
+  d1: 0.2500
+  r1: 0.4025
+  distinct_2: 0.7106
+  repetition_2: 0.0995
+  d2: 0.7106
+  r2: 0.0995
+  distinct_4: 0.9752
+  repetition_4: 0.0205
+  d4: 0.9752
+  r4: 0.0205
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.84_gpu0.json

eval/ar_baseline_latest_test/logs/temp0.84_gpu2.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.84)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'LONDON — The British government’s allowance for tax avoidance had been so underreported that in some cases they were so '
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 2.436140074805089,
+  "median_nll": 1.7247057557106018,
+  "ppl": 11.428841022454263,
+  "acc": 0.47911668951336533,
+  "tokens": 46688,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.84,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.84_gpu2.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.84_gpu2.json
+Diversity metrics for temp0.84_gpu2.json:
+  distinct_1: 0.2474
+  repetition_1: 0.4114
+  d1: 0.2474
+  r1: 0.4114
+  distinct_2: 0.7007
+  repetition_2: 0.1073
+  d2: 0.7007
+  r2: 0.1073
+  distinct_4: 0.9659
+  repetition_4: 0.0270
+  d4: 0.9659
+  r4: 0.0270
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.84_gpu2.json

eval/ar_baseline_latest_test/logs/temp0.84_gpu6.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.84)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'BETHLEHEM (Ma’an) -- The Palestinian Authority on Wednesday announced it was establishing a “protective force” in the We'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 2.484979679385866,
+  "median_nll": 1.7636990547180176,
+  "ppl": 12.000876387175099,
+  "acc": 0.47349288778505305,
+  "tokens": 44290,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.84,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.84_gpu6.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.84_gpu6.json
+Diversity metrics for temp0.84_gpu6.json:
+  distinct_1: 0.2487
+  repetition_1: 0.4072
+  d1: 0.2487
+  r1: 0.4072
+  distinct_2: 0.7112
+  repetition_2: 0.1041
+  d2: 0.7112
+  r2: 0.1041
+  distinct_4: 0.9685
+  repetition_4: 0.0285
+  d4: 0.9685
+  r4: 0.0285
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.84_gpu6.json

eval/ar_baseline_latest_test/logs/temp0.86_gpu0.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.86)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'The protagonist of the Möbius strip is a high school student who is teaching the world about the “Mokai,” the Japanese t'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 2.579709587299751,
+  "median_nll": 1.811776041984558,
+  "ppl": 13.193306099596239,
+  "acc": 0.46876446089773255,
+  "tokens": 45381,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.86,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.86_gpu0.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.86_gpu0.json
+Diversity metrics for temp0.86_gpu0.json:
+  distinct_1: 0.2587
+  repetition_1: 0.3975
+  d1: 0.2587
+  r1: 0.3975
+  distinct_2: 0.7234
+  repetition_2: 0.0981
+  d2: 0.7234
+  r2: 0.0981
+  distinct_4: 0.9763
+  repetition_4: 0.0233
+  d4: 0.9763
+  r4: 0.0233
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.86_gpu0.json

eval/ar_baseline_latest_test/logs/temp0.8_gpu3.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.8)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'Jürgen Klopp has hailed the work of the Liverpool Foundation, who are working to raise funds to help children in the are'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 2.2107951077390418,
+  "median_nll": 1.5452141761779785,
+  "ppl": 9.122967251916329,
+  "acc": 0.5083579377697025,
+  "tokens": 44030,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.8,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.8_gpu3.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.8_gpu3.json
+Diversity metrics for temp0.8_gpu3.json:
+  distinct_1: 0.2305
+  repetition_1: 0.4438
+  d1: 0.2305
+  r1: 0.4438
+  distinct_2: 0.6679
+  repetition_2: 0.1320
+  d2: 0.6679
+  r2: 0.1320
+  distinct_4: 0.9588
+  repetition_4: 0.0337
+  d4: 0.9588
+  r4: 0.0337
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.8_gpu3.json

eval/ar_baseline_latest_test/logs/temp0.8_gpu4.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.8)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'There is a lot of interest when it comes to the Blackhawks, and the World Series and Stanley Cup. Here’s a few great way'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 2.2282093542073826,
+  "median_nll": 1.5320324897766113,
+  "ppl": 9.283228214123909,
+  "acc": 0.5137793978787413,
+  "tokens": 45539,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.8,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.8_gpu4.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.8_gpu4.json
+Diversity metrics for temp0.8_gpu4.json:
+  distinct_1: 0.2408
+  repetition_1: 0.4306
+  d1: 0.2408
+  r1: 0.4306
+  distinct_2: 0.6806
+  repetition_2: 0.1304
+  d2: 0.6806
+  r2: 0.1304
+  distinct_4: 0.9530
+  repetition_4: 0.0372
+  d4: 0.9530
+  r4: 0.0372
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.8_gpu4.json

eval/ar_baseline_latest_test/logs/temp0.8_gpu7.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.8)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: '“Muppets” creators John DiLanzo and Gary Ungarick have announced that they will be returning to the studio with “Star Wa'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 2.2224382537774003,
+  "median_nll": 1.5243245959281921,
+  "ppl": 9.229808066569772,
+  "acc": 0.5135588247621443,
+  "tokens": 47718,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.8,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.8_gpu7.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.8_gpu7.json
+Diversity metrics for temp0.8_gpu7.json:
+  distinct_1: 0.2386
+  repetition_1: 0.4307
+  d1: 0.2386
+  r1: 0.4307
+  distinct_2: 0.6809
+  repetition_2: 0.1280
+  d2: 0.6809
+  r2: 0.1280
+  distinct_4: 0.9595
+  repetition_4: 0.0308
+  d4: 0.9595
+  r4: 0.0308
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.8_gpu7.json

eval/ar_baseline_latest_test/logs/temp0.92_gpu0.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.92)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'The protagonist? Angela Möller. The screenwriter?\n\nGlenn Tych at The Telegraph has had a chance to take a look at the sc'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 3.0223417018101086,
+  "median_nll": 2.2005038261413574,
+  "ppl": 20.53933240559053,
+  "acc": 0.41460409186513414,
+  "tokens": 47069,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.92,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.92_gpu0.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.92_gpu0.json
+Diversity metrics for temp0.92_gpu0.json:
+  distinct_1: 0.2825
+  repetition_1: 0.3581
+  d1: 0.2825
+  r1: 0.3581
+  distinct_2: 0.7673
+  repetition_2: 0.0632
+  d2: 0.7673
+  r2: 0.0632
+  distinct_4: 0.9886
+  repetition_4: 0.0095
+  d4: 0.9886
+  r4: 0.0095
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.92_gpu0.json

eval/ar_baseline_latest_test/logs/temp0.92_gpu6.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.92)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'BETHLEHEM (Ma’an) -- The Palestinian Authority on Wednesday announced it was establishing a “protective force” in the We'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 3.0061634738295786,
+  "median_nll": 2.166841506958008,
+  "ppl": 20.209715898184825,
+  "acc": 0.41916044648359924,
+  "tokens": 43809,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.92,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.92_gpu6.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.92_gpu6.json
+Diversity metrics for temp0.92_gpu6.json:
+  distinct_1: 0.2937
+  repetition_1: 0.3543
+  d1: 0.2937
+  r1: 0.3543
+  distinct_2: 0.7686
+  repetition_2: 0.0697
+  d2: 0.7686
+  r2: 0.0697
+  distinct_4: 0.9845
+  repetition_4: 0.0134
+  d4: 0.9845
+  r4: 0.0134
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.92_gpu6.json

eval/ar_baseline_latest_test/logs/temp0.94_gpu0.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.94)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'The protagonist? Angela Möttins, better known as "Griezmann" at Real Madrid.\n\nIn reality, this role in the history of th'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 3.186605011933174,
+  "median_nll": 2.3521339893341064,
+  "ppl": 24.20610833702178,
+  "acc": 0.39720741418151806,
+  "tokens": 44833,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.94,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.94_gpu0.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.94_gpu0.json
+Diversity metrics for temp0.94_gpu0.json:
+  distinct_1: 0.2940
+  repetition_1: 0.3367
+  d1: 0.2940
+  r1: 0.3367
+  distinct_2: 0.7849
+  repetition_2: 0.0527
+  d2: 0.7849
+  r2: 0.0527
+  distinct_4: 0.9901
+  repetition_4: 0.0084
+  d4: 0.9901
+  r4: 0.0084
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.94_gpu0.json

eval/ar_baseline_latest_test/logs/temp0.94_gpu2.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.94)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'LONDON — The British media reported in articles Wednesday that ISIS has taken over Kirkuk, the Iraqi town that is disput'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 3.175500140291807,
+  "median_nll": 2.341375946998596,
+  "ppl": 23.93878962807616,
+  "acc": 0.3984287317620651,
+  "tokens": 44550,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.94,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.94_gpu2.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.94_gpu2.json
+Diversity metrics for temp0.94_gpu2.json:
+  distinct_1: 0.2911
+  repetition_1: 0.3433
+  d1: 0.2911
+  r1: 0.3433
+  distinct_2: 0.7821
+  repetition_2: 0.0540
+  d2: 0.7821
+  r2: 0.0540
+  distinct_4: 0.9914
+  repetition_4: 0.0063
+  d4: 0.9914
+  r4: 0.0063
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.94_gpu2.json

eval/ar_baseline_latest_test/logs/temp0.94_gpu5.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.94)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'Bless up! So happy to be back to where I started!I want to thank yoeoi for the email comment taking me back to when I go'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 3.2659872423456764,
+  "median_nll": 2.4272470474243164,
+  "ppl": 26.205969867396313,
+  "acc": 0.39182821610689156,
+  "tokens": 46477,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.94,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.94_gpu5.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.94_gpu5.json
+Diversity metrics for temp0.94_gpu5.json:
+  distinct_1: 0.2964
+  repetition_1: 0.3412
+  d1: 0.2964
+  r1: 0.3412
+  distinct_2: 0.7810
+  repetition_2: 0.0550
+  d2: 0.7810
+  r2: 0.0550
+  distinct_4: 0.9902
+  repetition_4: 0.0068
+  d4: 0.9902
+  r4: 0.0068
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.94_gpu5.json

eval/ar_baseline_latest_test/logs/temp0.94_gpu6.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.94)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'BETHLEHEM (Ma’an) -- The Palestinian Authority on Wednesday announced it was establishing a “protective force” in the We'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 3.192733724355068,
+  "median_nll": 2.3344547748565674,
+  "ppl": 24.354916148231116,
+  "acc": 0.4002527817274923,
+  "tokens": 44307,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.94,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.94_gpu6.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.94_gpu6.json
+Diversity metrics for temp0.94_gpu6.json:
+  distinct_1: 0.3008
+  repetition_1: 0.3345
+  d1: 0.3008
+  r1: 0.3345
+  distinct_2: 0.7881
+  repetition_2: 0.0529
+  d2: 0.7881
+  r2: 0.0529
+  distinct_4: 0.9895
+  repetition_4: 0.0072
+  d4: 0.9895
+  r4: 0.0072
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.94_gpu6.json

eval/ar_baseline_latest_test/logs/temp0.96_gpu7.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.96)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: '“Muppets” creators William And Bryan Thompson and Coleman Hentzen are perhaps the biggest Javascript evangelists you won'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 3.347551785956083,
+  "median_nll": 2.4971165657043457,
+  "ppl": 28.43303820046356,
+  "acc": 0.3820270361761188,
+  "tokens": 44311,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.96,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.96_gpu7.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.96_gpu7.json
+Diversity metrics for temp0.96_gpu7.json:
+  distinct_1: 0.3048
+  repetition_1: 0.3282
+  d1: 0.3048
+  r1: 0.3282
+  distinct_2: 0.7967
+  repetition_2: 0.0504
+  d2: 0.7967
+  r2: 0.0504
+  distinct_4: 0.9934
+  repetition_4: 0.0056
+  d4: 0.9934
+  r4: 0.0056
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.96_gpu7.json

eval/ar_baseline_latest_test/logs/temp0.98_gpu1.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.98)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: "First and foremost we'd like to thank everyone who participated in the Equestrian/SFX / Rock ’Em Puppy contest and those"
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 3.492161918339162,
+  "median_nll": 2.6467158794403076,
+  "ppl": 32.8569049220607,
+  "acc": 0.37063684761544113,
+  "tokens": 44893,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.98,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.98_gpu1.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.98_gpu1.json
+Diversity metrics for temp0.98_gpu1.json:
+  distinct_1: 0.3052
+  repetition_1: 0.3218
+  d1: 0.3052
+  r1: 0.3218
+  distinct_2: 0.7911
+  repetition_2: 0.0527
+  d2: 0.7911
+  r2: 0.0527
+  distinct_4: 0.9768
+  repetition_4: 0.0154
+  d4: 0.9768
+  r4: 0.0154
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.98_gpu1.json

eval/ar_baseline_latest_test/logs/temp0.98_gpu2.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.98)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'LONDON — The British media reported in articles Wednesday that ISIS has taken over Kirkuk, the Iraqi town that is disput'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 3.5845073000923775,
+  "median_nll": 2.740861415863037,
+  "ppl": 36.03559860815703,
+  "acc": 0.3579749002996643,
+  "tokens": 44383,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.98,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.98_gpu2.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.98_gpu2.json
+Diversity metrics for temp0.98_gpu2.json:
+  distinct_1: 0.3213
+  repetition_1: 0.3055
+  d1: 0.3213
+  r1: 0.3055
+  distinct_2: 0.8150
+  repetition_2: 0.0394
+  d2: 0.8150
+  r2: 0.0394
+  distinct_4: 0.9945
+  repetition_4: 0.0031
+  d4: 0.9945
+  r4: 0.0031
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.98_gpu2.json

eval/ar_baseline_latest_test/logs/temp0.9_gpu2.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.9)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'LONDON — The British media reported in articles Wednesday that the New York-based investment fund founded in London by e'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 2.8760180329326013,
+  "median_nll": 2.0924065113067627,
+  "ppl": 17.74347837531484,
+  "acc": 0.4278127152729939,
+  "tokens": 45001,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.9,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.9_gpu2.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.9_gpu2.json
+Diversity metrics for temp0.9_gpu2.json:
+  distinct_1: 0.2695
+  repetition_1: 0.3647
+  d1: 0.2695
+  r1: 0.3647
+  distinct_2: 0.7520
+  repetition_2: 0.0685
+  d2: 0.7520
+  r2: 0.0685
+  distinct_4: 0.9846
+  repetition_4: 0.0106
+  d4: 0.9846
+  r4: 0.0106
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.9_gpu2.json

eval/ar_baseline_latest_test/logs/temp0.9_gpu4.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=0.9)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: 'There is much heroism in the history of theatre in the United States. Whom to thank for this? Civil rights activist and '
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 2.866953418245275,
+  "median_nll": 2.039194107055664,
+  "ppl": 17.58336734933049,
+  "acc": 0.4339618499901223,
+  "tokens": 45557,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 0.9,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp0.9_gpu4.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp0.9_gpu4.json
+Diversity metrics for temp0.9_gpu4.json:
+  distinct_1: 0.2818
+  repetition_1: 0.3673
+  d1: 0.2818
+  r1: 0.3673
+  distinct_2: 0.7574
+  repetition_2: 0.0721
+  d2: 0.7574
+  r2: 0.0721
+  distinct_4: 0.9824
+  repetition_4: 0.0126
+  d4: 0.9824
+  r4: 0.0126
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp0.9_gpu4.json

eval/ar_baseline_latest_test/logs/temp1_gpu7.log

@@ -0,0 +1,50 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt
+Loaded AR checkpoint: /home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt (step=500000)
+Generating 128 samples (seq_len=512, temperature=1.0)...
+  sampled 64/128
+  sampled 128/128
+First sample preview: '“Muppets” creators William And Bryan Thompson and Coleman Hentzen are perhaps the biggest Javascript evangelists you won'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 3.868952017700726,
+  "median_nll": 3.053391456604004,
+  "ppl": 47.892169626395194,
+  "acc": 0.3375880753003662,
+  "tokens": 46693,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/ar_baseline/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "ar_generation",
+  "temperature": 1.0,
+  "top_k": 0,
+  "top_p": 1.0,
+  "prompt_len": 1,
+  "max_new_tokens": 511,
+  "stop_on_eos": false
+}
+Saved metrics -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/metrics/temp1_gpu7.json
+Saved samples  -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/samples/temp1_gpu7.json
+Diversity metrics for temp1_gpu7.json:
+  distinct_1: 0.3382
+  repetition_1: 0.2992
+  d1: 0.3382
+  r1: 0.2992
+  distinct_2: 0.8313
+  repetition_2: 0.0370
+  d2: 0.8313
+  r2: 0.0370
+  distinct_4: 0.9945
+  repetition_4: 0.0036
+  d4: 0.9945
+  r4: 0.0036
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/ar_baseline_latest_test/diversity/temp1_gpu7.json

eval/h2_mixed_latest_test/logs/pps1_lam0.2_temp0.98_gpu4.log

@@ -0,0 +1,53 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from cli:/home/sunhc/diffusion/sad/configs/block_ar_owt_b32_h2_mixed.yaml
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.2, 0.2]
+leaf_temperature = 0.98
+Generating 128 samples (L=512)...
+  sampled 128/128 (steps this call: 871)
+Average denoising rounds per sample: 871.00
+First sample preview: '. "It was like it was the alleged victim at night. It was so dark, and as the computers woke up -- having really shut up'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 5.002073652772666,
+  "median_nll": 4.476129055023193,
+  "ppl": 148.72123577307576,
+  "acc": 0.2389584100110416,
+  "tokens": 65208,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "model_type": "block",
+  "level_lambdas": "0.2,0.2",
+  "avg_steps": 871.0,
+  "positions_per_step": 1,
+  "leaf_temperature": 0.98
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps1_lam0.2_temp0.98_gpu4.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps1_lam0.2_temp0.98_gpu4.json
+Diversity metrics for pps1_lam0.2_temp0.98_gpu4.json:
+  distinct_1: 0.3263
+  repetition_1: 0.3001
+  d1: 0.3263
+  r1: 0.3001
+  distinct_2: 0.8356
+  repetition_2: 0.0237
+  d2: 0.8356
+  r2: 0.0237
+  distinct_4: 0.9988
+  repetition_4: 0.0006
+  d4: 0.9988
+  r4: 0.0006
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps1_lam0.2_temp0.98_gpu4.json

eval/h2_mixed_latest_test/logs/pps1_lam0.2_temp1_gpu6.log

@@ -0,0 +1,53 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from cli:/home/sunhc/diffusion/sad/configs/block_ar_owt_b32_h2_mixed.yaml
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.2, 0.2]
+leaf_temperature = 1.0
+Generating 128 samples (L=512)...
+  sampled 128/128 (steps this call: 953)
+Average denoising rounds per sample: 953.00
+First sample preview: ' to point to Twitter will help too.\n\n20: The CW •• DC • Eleventh\u200bArrow plus CW\n\nTriDest does have an expanded look withi'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 5.225233735122243,
+  "median_nll": 4.764096021652222,
+  "ppl": 185.9046178867565,
+  "acc": 0.21908647504524678,
+  "tokens": 65198,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "model_type": "block",
+  "level_lambdas": "0.2,0.2",
+  "avg_steps": 953.0,
+  "positions_per_step": 1,
+  "leaf_temperature": 1.0
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps1_lam0.2_temp1_gpu6.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps1_lam0.2_temp1_gpu6.json
+Diversity metrics for pps1_lam0.2_temp1_gpu6.json:
+  distinct_1: 0.3313
+  repetition_1: 0.3016
+  d1: 0.3313
+  r1: 0.3016
+  distinct_2: 0.8383
+  repetition_2: 0.0243
+  d2: 0.8383
+  r2: 0.0243
+  distinct_4: 0.9984
+  repetition_4: 0.0009
+  d4: 0.9984
+  r4: 0.0009
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps1_lam0.2_temp1_gpu6.json

eval/h2_mixed_latest_test/logs/pps1_lam0.2_temp1_gpu7.log

@@ -0,0 +1,53 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from cli:/home/sunhc/diffusion/sad/configs/block_ar_owt_b32_h2_mixed.yaml
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.2, 0.2]
+leaf_temperature = 1.0
+Generating 128 samples (L=512)...
+  sampled 128/128 (steps this call: 939)
+Average denoising rounds per sample: 939.00
+First sample preview: 's her resounding expedition to fuel use failed to kill the house, which eloquently echoes a notorious perhaps like Frank'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 5.26980805105218,
+  "median_nll": 4.809404373168945,
+  "ppl": 194.3786480873024,
+  "acc": 0.21971532869106414,
+  "tokens": 65198,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "model_type": "block",
+  "level_lambdas": "0.2,0.2",
+  "avg_steps": 939.0,
+  "positions_per_step": 1,
+  "leaf_temperature": 1.0
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps1_lam0.2_temp1_gpu7.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps1_lam0.2_temp1_gpu7.json
+Diversity metrics for pps1_lam0.2_temp1_gpu7.json:
+  distinct_1: 0.3403
+  repetition_1: 0.2934
+  d1: 0.3403
+  r1: 0.2934
+  distinct_2: 0.8462
+  repetition_2: 0.0225
+  d2: 0.8462
+  r2: 0.0225
+  distinct_4: 0.9992
+  repetition_4: 0.0003
+  d4: 0.9992
+  r4: 0.0003
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps1_lam0.2_temp1_gpu7.json

eval/h2_mixed_latest_test/logs/pps1_lam0.3_temp0.8_gpu1.log

@@ -0,0 +1,52 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from cli:/home/sunhc/diffusion/sad/configs/block_ar_owt_b32_h2_mixed.yaml
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.3, 0.3]
+leaf_temperature = 0.8
+Generating 128 samples (L=512)...
+  sampled 128/128 (steps this call: 736)
+Average denoising rounds per sample: 736.00
+First sample preview: ' 3, 4, 7 3, 2, 8, 5, 7 3, 3, 3, 4 3, 3, 5 3, 3, 3 3, 3, 8, 7 3, 3, 6, 5 3, 3, 6, 6 3, 3, 3, 3 7 3, 5 4, 1, 2, 1, 7 5, 1,'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 3.1288898825283176,
+  "median_nll": 2.3229328393936157,
+  "ppl": 22.848600827387646,
+  "acc": 0.39922337845719374,
+  "tokens": 65154,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "level_lambdas": "0.3,0.3",
+  "avg_steps": 736.0,
+  "positions_per_step": 1,
+  "leaf_temperature": 0.8
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps1_lam0.3_temp0.8_gpu1.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps1_lam0.3_temp0.8_gpu1.json
+Diversity metrics for pps1_lam0.3_temp0.8_gpu1.json:
+  distinct_1: 0.2121
+  repetition_1: 0.4603
+  d1: 0.2121
+  r1: 0.4603
+  distinct_2: 0.6760
+  repetition_2: 0.1178
+  d2: 0.6760
+  r2: 0.1178
+  distinct_4: 0.9735
+  repetition_4: 0.0227
+  d4: 0.9735
+  r4: 0.0227
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps1_lam0.3_temp0.8_gpu1.json

eval/h2_mixed_latest_test/logs/pps1_lam0.3_temp0.8_gpu3.log

@@ -0,0 +1,52 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from cli:/home/sunhc/diffusion/sad/configs/block_ar_owt_b32_h2_mixed.yaml
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.3, 0.3]
+leaf_temperature = 0.8
+Generating 128 samples (L=512)...
+  sampled 128/128 (steps this call: 753)
+Average denoising rounds per sample: 753.00
+First sample preview: ' sure that nobody covers them.\n\nAdvertisement\n\nThey chanted and sang for the national anthem and the Astoria State anthe'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 3.202027189317,
+  "median_nll": 2.4177517890930176,
+  "ppl": 24.582312722497303,
+  "acc": 0.3893960081616372,
+  "tokens": 65183,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "level_lambdas": "0.3,0.3",
+  "avg_steps": 753.0,
+  "positions_per_step": 1,
+  "leaf_temperature": 0.8
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps1_lam0.3_temp0.8_gpu3.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps1_lam0.3_temp0.8_gpu3.json
+Diversity metrics for pps1_lam0.3_temp0.8_gpu3.json:
+  distinct_1: 0.2134
+  repetition_1: 0.4545
+  d1: 0.2134
+  r1: 0.4545
+  distinct_2: 0.6809
+  repetition_2: 0.1049
+  d2: 0.6809
+  r2: 0.1049
+  distinct_4: 0.9807
+  repetition_4: 0.0124
+  d4: 0.9807
+  r4: 0.0124
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps1_lam0.3_temp0.8_gpu3.json

eval/h2_mixed_latest_test/logs/pps1_lam0.4_temp0.8_gpu0.log

@@ -0,0 +1,40 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.4, 0.4]
+leaf_temperature = 0.8
+Generating 128 samples (L=512)...
+  sampled 32/128 (steps this call: 778)
+  sampled 64/128 (steps this call: 812)
+  sampled 96/128 (steps this call: 821)
+  sampled 128/128 (steps this call: 832)
+Average denoising rounds per sample: 810.75
+First sample preview: 'Australians hate Kim Shau, a social media user who would like to say: "Oh no, only rich people are waiting for me", says'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 3.2707012796030432,
+  "median_nll": 2.45578670501709,
+  "ppl": 26.329797420256064,
+  "acc": 0.3880606172157801,
+  "tokens": 65196,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "level_lambdas": "0.4,0.4",
+  "avg_steps": 810.75,
+  "positions_per_step": 1,
+  "leaf_temperature": 0.8
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps1_lam0.4_temp0.8_gpu0.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps1_lam0.4_temp0.8_gpu0.json

eval/h2_mixed_latest_test/logs/pps1_lam0.6_temp0.98_gpu4.log

@@ -0,0 +1,53 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from cli:/home/sunhc/diffusion/sad/configs/block_ar_owt_b32_h2_mixed.yaml
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.6, 0.6]
+leaf_temperature = 0.98
+Generating 128 samples (L=512)...
+  sampled 128/128 (steps this call: 1333)
+Average denoising rounds per sample: 1333.00
+First sample preview: '. "It was clear what came out of the scorpion. We ran a tar, tape and a pencil as it goes but we have our first nail cut'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 4.9946654149401475,
+  "median_nll": 4.464876890182495,
+  "ppl": 147.62354448706483,
+  "acc": 0.2424953959484346,
+  "tokens": 65160,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "model_type": "block",
+  "level_lambdas": "0.6,0.6",
+  "avg_steps": 1333.0,
+  "positions_per_step": 1,
+  "leaf_temperature": 0.98
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps1_lam0.6_temp0.98_gpu4.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps1_lam0.6_temp0.98_gpu4.json
+Diversity metrics for pps1_lam0.6_temp0.98_gpu4.json:
+  distinct_1: 0.3372
+  repetition_1: 0.3038
+  d1: 0.3372
+  r1: 0.3038
+  distinct_2: 0.8377
+  repetition_2: 0.0283
+  d2: 0.8377
+  r2: 0.0283
+  distinct_4: 0.9982
+  repetition_4: 0.0012
+  d4: 0.9982
+  r4: 0.0012
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps1_lam0.6_temp0.98_gpu4.json

eval/h2_mixed_latest_test/logs/pps1_lam0.6_temp0.9_gpu5.log

@@ -0,0 +1,55 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.6, 0.6]
+leaf_temperature = 0.9
+Generating 128 samples (L=512)...
+  sampled 32/128 (steps this call: 1105)
+  sampled 64/128 (steps this call: 1157)
+  sampled 96/128 (steps this call: 1120)
+  sampled 128/128 (steps this call: 1127)
+Average denoising rounds per sample: 1127.25
+First sample preview: ' hemp.\n\nLegal\n\nThe growing market for hemp has drummed up goodwill. According to its website, the American Dictionary of'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 4.223532691408287,
+  "median_nll": 3.5249366760253906,
+  "ppl": 68.27425062102515,
+  "acc": 0.29709878247002175,
+  "tokens": 65214,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "level_lambdas": "0.6,0.6",
+  "avg_steps": 1127.25,
+  "positions_per_step": 1,
+  "leaf_temperature": 0.9
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps1_lam0.6_temp0.9_gpu5.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps1_lam0.6_temp0.9_gpu5.json
+Diversity metrics for pps1_lam0.6_temp0.9_gpu5.json:
+  distinct_1: 0.2830
+  repetition_1: 0.3659
+  d1: 0.2830
+  r1: 0.3659
+  distinct_2: 0.7855
+  repetition_2: 0.0505
+  d2: 0.7855
+  r2: 0.0505
+  distinct_4: 0.9954
+  repetition_4: 0.0029
+  d4: 0.9954
+  r4: 0.0029
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps1_lam0.6_temp0.9_gpu5.json

eval/h2_mixed_latest_test/logs/pps1_lam0.7_temp0.9_gpu0.log

@@ -0,0 +1,51 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.7, 0.7]
+leaf_temperature = 0.9
+Generating 128 samples (L=512)...
+  sampled 64/128 (steps this call: 1215)
+  sampled 128/128 (steps this call: 1198)
+Average denoising rounds per sample: 1206.50
+First sample preview: "'s why [the] state's like destabilizing it. I remember [the state] had a 100-page list of nuclear weapons and they didn'"
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 4.1800211892045365,
+  "median_nll": 3.4635732173919678,
+  "ppl": 65.36723827911753,
+  "acc": 0.30498641742253324,
+  "tokens": 65157,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "level_lambdas": "0.7,0.7",
+  "avg_steps": 1206.5,
+  "positions_per_step": 1,
+  "leaf_temperature": 0.9
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps1_lam0.7_temp0.9_gpu0.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps1_lam0.7_temp0.9_gpu0.json
+Diversity metrics for pps1_lam0.7_temp0.9_gpu0.json:
+  distinct_1: 0.2883
+  repetition_1: 0.3510
+  d1: 0.2883
+  r1: 0.3510
+  distinct_2: 0.7919
+  repetition_2: 0.0445
+  d2: 0.7919
+  r2: 0.0445
+  distinct_4: 0.9962
+  repetition_4: 0.0023
+  d4: 0.9962
+  r4: 0.0023
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps1_lam0.7_temp0.9_gpu0.json

eval/h2_mixed_latest_test/logs/pps1_lam0_temp0.7_gpu1.log

@@ -0,0 +1,55 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.0, 0.0]
+leaf_temperature = 0.7
+Generating 128 samples (L=512)...
+  sampled 32/128 (steps this call: 512)
+  sampled 64/128 (steps this call: 512)
+  sampled 96/128 (steps this call: 512)
+  sampled 128/128 (steps this call: 512)
+Average denoising rounds per sample: 512.00
+First sample preview: 'Hello Military Users,\n\n---\n\nNote: Some want to quote us from:\n\nAs of last Thursday, we went to work on the new engine us'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 2.4444901757231374,
+  "median_nll": 1.648135781288147,
+  "ppl": 11.524672542987966,
+  "acc": 0.4860126989969633,
+  "tokens": 65202,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "level_lambdas": "0,0",
+  "avg_steps": 512.0,
+  "positions_per_step": 1,
+  "leaf_temperature": 0.7
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps1_lam0_temp0.7_gpu1.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps1_lam0_temp0.7_gpu1.json
+Diversity metrics for pps1_lam0_temp0.7_gpu1.json:
+  distinct_1: 0.1709
+  repetition_1: 0.5429
+  d1: 0.1709
+  r1: 0.5429
+  distinct_2: 0.5770
+  repetition_2: 0.2116
+  d2: 0.5770
+  r2: 0.2116
+  distinct_4: 0.9309
+  repetition_4: 0.0593
+  d4: 0.9309
+  r4: 0.0593
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps1_lam0_temp0.7_gpu1.json

eval/h2_mixed_latest_test/logs/pps2_lam0.2_temp1_gpu1.log

@@ -0,0 +1,53 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from cli:/home/sunhc/diffusion/sad/configs/block_ar_owt_b32_h2_mixed.yaml
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.2, 0.2]
+leaf_temperature = 1.0
+Generating 128 samples (L=512)...
+  sampled 128/128 (steps this call: 512)
+Average denoising rounds per sample: 512.00
+First sample preview: ' and donate to a grassroots movement\n\nin Bali municipalities,\nweijmas Campaign for demolition of 3rd house and stone ent'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 5.675843099708208,
+  "median_nll": 5.313490867614746,
+  "ppl": 291.73419590506444,
+  "acc": 0.19218306073869307,
+  "tokens": 65115,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "model_type": "block",
+  "level_lambdas": "0.2,0.2",
+  "avg_steps": 512.0,
+  "positions_per_step": 2,
+  "leaf_temperature": 1.0
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps2_lam0.2_temp1_gpu1.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps2_lam0.2_temp1_gpu1.json
+Diversity metrics for pps2_lam0.2_temp1_gpu1.json:
+  distinct_1: 0.3569
+  repetition_1: 0.2830
+  d1: 0.3569
+  r1: 0.2830
+  distinct_2: 0.8555
+  repetition_2: 0.0209
+  d2: 0.8555
+  r2: 0.0209
+  distinct_4: 0.9994
+  repetition_4: 0.0001
+  d4: 0.9994
+  r4: 0.0001
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps2_lam0.2_temp1_gpu1.json

eval/h2_mixed_latest_test/logs/pps2_lam0.3_temp0.8_gpu4.log

@@ -0,0 +1,52 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from cli:/home/sunhc/diffusion/sad/configs/block_ar_owt_b32_h2_mixed.yaml
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.3, 0.3]
+leaf_temperature = 0.8
+Generating 128 samples (L=512)...
+  sampled 128/128 (steps this call: 387)
+Average denoising rounds per sample: 387.00
+First sample preview: '"It was like we gave up," he said.\n\nThe residents of the tower declined to comment for the story, but have pointed to th'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 3.482659498755244,
+  "median_nll": 2.7303624153137207,
+  "ppl": 32.54616356088146,
+  "acc": 0.3610560447497426,
+  "tokens": 65073,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "level_lambdas": "0.3,0.3",
+  "avg_steps": 387.0,
+  "positions_per_step": 2,
+  "leaf_temperature": 0.8
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps2_lam0.3_temp0.8_gpu4.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps2_lam0.3_temp0.8_gpu4.json
+Diversity metrics for pps2_lam0.3_temp0.8_gpu4.json:
+  distinct_1: 0.2266
+  repetition_1: 0.4379
+  d1: 0.2266
+  r1: 0.4379
+  distinct_2: 0.7021
+  repetition_2: 0.0890
+  d2: 0.7021
+  r2: 0.0890
+  distinct_4: 0.9878
+  repetition_4: 0.0079
+  d4: 0.9878
+  r4: 0.0079
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps2_lam0.3_temp0.8_gpu4.json

eval/h2_mixed_latest_test/logs/pps2_lam0.3_temp0.9_gpu4.log

@@ -0,0 +1,51 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.3, 0.3]
+leaf_temperature = 0.9
+Generating 128 samples (L=512)...
+  sampled 64/128 (steps this call: 453)
+  sampled 128/128 (steps this call: 454)
+Average denoising rounds per sample: 453.50
+First sample preview: ' that rather than being helped by others like Manning’s chronic lack of eagerness.\n\nThat said, the Broncos need a dire c'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 4.41296852458639,
+  "median_nll": 3.775259852409363,
+  "ppl": 82.514045273213,
+  "acc": 0.27863040608981243,
+  "tokens": 65158,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "level_lambdas": "0.3,0.3",
+  "avg_steps": 453.5,
+  "positions_per_step": 2,
+  "leaf_temperature": 0.9
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps2_lam0.3_temp0.9_gpu4.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps2_lam0.3_temp0.9_gpu4.json
+Diversity metrics for pps2_lam0.3_temp0.9_gpu4.json:
+  distinct_1: 0.2770
+  repetition_1: 0.3646
+  d1: 0.2770
+  r1: 0.3646
+  distinct_2: 0.7787
+  repetition_2: 0.0444
+  d2: 0.7787
+  r2: 0.0444
+  distinct_4: 0.9964
+  repetition_4: 0.0013
+  d4: 0.9964
+  r4: 0.0013
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps2_lam0.3_temp0.9_gpu4.json

eval/h2_mixed_latest_test/logs/pps2_lam0.6_temp0.94_gpu3.log

@@ -0,0 +1,53 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from cli:/home/sunhc/diffusion/sad/configs/block_ar_owt_b32_h2_mixed.yaml
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.6, 0.6]
+leaf_temperature = 0.94
+Generating 128 samples (L=512)...
+  sampled 128/128 (steps this call: 652)
+Average denoising rounds per sample: 652.00
+First sample preview: ' Minutearo Minus.\n\n9,8 Marcel Jr.\n\n9\n\nmolonio Junior nabbed a replay, adding 1 TOV to the game in a row.\n\nJust 2 percent'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 4.835658662038316,
+  "median_nll": 4.23482084274292,
+  "ppl": 125.9214956072145,
+  "acc": 0.25355377483037056,
+  "tokens": 65142,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "model_type": "block",
+  "level_lambdas": "0.6,0.6",
+  "avg_steps": 652.0,
+  "positions_per_step": 2,
+  "leaf_temperature": 0.94
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps2_lam0.6_temp0.94_gpu3.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps2_lam0.6_temp0.94_gpu3.json
+Diversity metrics for pps2_lam0.6_temp0.94_gpu3.json:
+  distinct_1: 0.3206
+  repetition_1: 0.3197
+  d1: 0.3206
+  r1: 0.3197
+  distinct_2: 0.8238
+  repetition_2: 0.0345
+  d2: 0.8238
+  r2: 0.0345
+  distinct_4: 0.9980
+  repetition_4: 0.0011
+  d4: 0.9980
+  r4: 0.0011
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps2_lam0.6_temp0.94_gpu3.json

eval/h2_mixed_latest_test/logs/pps2_lam0.6_temp1_gpu5.log

@@ -0,0 +1,53 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from cli:/home/sunhc/diffusion/sad/configs/block_ar_owt_b32_h2_mixed.yaml
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.6, 0.6]
+leaf_temperature = 1.0
+Generating 128 samples (L=512)...
+  sampled 128/128 (steps this call: 719)
+Average denoising rounds per sample: 719.00
+First sample preview: ' the quasi-Suicide template.\n\nEvery surrogate of the Australian “killer cop” provided by Spox (whose cops were eclipsed '
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 5.496910366953119,
+  "median_nll": 5.08622932434082,
+  "ppl": 243.9370906762153,
+  "acc": 0.20297705823678355,
+  "tokens": 65165,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "model_type": "block",
+  "level_lambdas": "0.6,0.6",
+  "avg_steps": 719.0,
+  "positions_per_step": 2,
+  "leaf_temperature": 1.0
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps2_lam0.6_temp1_gpu5.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps2_lam0.6_temp1_gpu5.json
+Diversity metrics for pps2_lam0.6_temp1_gpu5.json:
+  distinct_1: 0.3585
+  repetition_1: 0.2841
+  d1: 0.3585
+  r1: 0.2841
+  distinct_2: 0.8633
+  repetition_2: 0.0215
+  d2: 0.8633
+  r2: 0.0215
+  distinct_4: 0.9991
+  repetition_4: 0.0005
+  d4: 0.9991
+  r4: 0.0005
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps2_lam0.6_temp1_gpu5.json

eval/h2_mixed_latest_test/logs/pps2_lam0.6_temp1_gpu6.log

@@ -0,0 +1,53 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from cli:/home/sunhc/diffusion/sad/configs/block_ar_owt_b32_h2_mixed.yaml
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.6, 0.6]
+leaf_temperature = 1.0
+Generating 128 samples (L=512)...
+  sampled 128/128 (steps this call: 703)
+Average denoising rounds per sample: 703.00
+First sample preview: ' -- I had the [Obama] children on that plane, they really have to." He offered recollections about the Romney\'s White Ho'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 5.530916663150708,
+  "median_nll": 5.155064582824707,
+  "ppl": 252.3751480224496,
+  "acc": 0.20308688381227083,
+  "tokens": 65179,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "model_type": "block",
+  "level_lambdas": "0.6,0.6",
+  "avg_steps": 703.0,
+  "positions_per_step": 2,
+  "leaf_temperature": 1.0
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps2_lam0.6_temp1_gpu6.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps2_lam0.6_temp1_gpu6.json
+Diversity metrics for pps2_lam0.6_temp1_gpu6.json:
+  distinct_1: 0.3621
+  repetition_1: 0.2806
+  d1: 0.3621
+  r1: 0.2806
+  distinct_2: 0.8631
+  repetition_2: 0.0215
+  d2: 0.8631
+  r2: 0.0215
+  distinct_4: 0.9989
+  repetition_4: 0.0005
+  d4: 0.9989
+  r4: 0.0005
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps2_lam0.6_temp1_gpu6.json

eval/h2_mixed_latest_test/logs/pps2_lam0.7_temp0.9_gpu6.log

@@ -0,0 +1,51 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.7, 0.7]
+leaf_temperature = 0.9
+Generating 128 samples (L=512)...
+  sampled 64/128 (steps this call: 620)
+  sampled 128/128 (steps this call: 622)
+Average denoising rounds per sample: 621.00
+First sample preview: 'Earlier this week, father of two Supreme Court Justices, S. Humayun (also known as the father of equal marriage) was ask'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 4.467391929840715,
+  "median_nll": 3.795100212097168,
+  "ppl": 87.1291873943412,
+  "acc": 0.28252462940797346,
+  "tokens": 65166,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "level_lambdas": "0.7,0.7",
+  "avg_steps": 621.0,
+  "positions_per_step": 2,
+  "leaf_temperature": 0.9
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps2_lam0.7_temp0.9_gpu6.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps2_lam0.7_temp0.9_gpu6.json
+Diversity metrics for pps2_lam0.7_temp0.9_gpu6.json:
+  distinct_1: 0.3043
+  repetition_1: 0.3403
+  d1: 0.3043
+  r1: 0.3403
+  distinct_2: 0.8084
+  repetition_2: 0.0394
+  d2: 0.8084
+  r2: 0.0394
+  distinct_4: 0.9975
+  repetition_4: 0.0012
+  d4: 0.9975
+  r4: 0.0012
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps2_lam0.7_temp0.9_gpu6.json

eval/h2_mixed_latest_test/logs/pps2_lam0.8_temp0.7_gpu4.log

@@ -0,0 +1,51 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.8, 0.8]
+leaf_temperature = 0.7
+Generating 128 samples (L=512)...
+  sampled 64/128 (steps this call: 508)
+  sampled 128/128 (steps this call: 505)
+Average denoising rounds per sample: 506.50
+First sample preview: ' stage at the newly opened Lucas and Mark Parker’s The Tour of Owings.\n\nI arrived in Raleigh to celebrate The Tour and t'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 3.015721177860601,
+  "median_nll": 2.095865249633789,
+  "ppl": 20.40380040496583,
+  "acc": 0.4302159379082456,
+  "tokens": 65065,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "level_lambdas": "0.8,0.8",
+  "avg_steps": 506.5,
+  "positions_per_step": 2,
+  "leaf_temperature": 0.7
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps2_lam0.8_temp0.7_gpu4.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps2_lam0.8_temp0.7_gpu4.json
+Diversity metrics for pps2_lam0.8_temp0.7_gpu4.json:
+  distinct_1: 0.2162
+  repetition_1: 0.4844
+  d1: 0.2162
+  r1: 0.4844
+  distinct_2: 0.6706
+  repetition_2: 0.1492
+  d2: 0.6706
+  r2: 0.1492
+  distinct_4: 0.9678
+  repetition_4: 0.0299
+  d4: 0.9678
+  r4: 0.0299
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps2_lam0.8_temp0.7_gpu4.json

eval/h2_mixed_latest_test/logs/pps2_lam0.8_temp0.94_gpu1.log

@@ -0,0 +1,53 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from cli:/home/sunhc/diffusion/sad/configs/block_ar_owt_b32_h2_mixed.yaml
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.8, 0.8]
+leaf_temperature = 0.94
+Generating 128 samples (L=512)...
+  sampled 128/128 (steps this call: 695)
+Average denoising rounds per sample: 695.00
+First sample preview: 'A young Australian citizen has been taken to a police station in Christchurch, charged with simple theft of his identity'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 4.888195943640045,
+  "median_nll": 4.315791606903076,
+  "ppl": 132.7139345370263,
+  "acc": 0.2462536619783119,
+  "tokens": 65197,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "model_type": "block",
+  "level_lambdas": "0.8,0.8",
+  "avg_steps": 695.0,
+  "positions_per_step": 2,
+  "leaf_temperature": 0.94
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps2_lam0.8_temp0.94_gpu1.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps2_lam0.8_temp0.94_gpu1.json
+Diversity metrics for pps2_lam0.8_temp0.94_gpu1.json:
+  distinct_1: 0.3206
+  repetition_1: 0.3230
+  d1: 0.3206
+  r1: 0.3230
+  distinct_2: 0.8286
+  repetition_2: 0.0332
+  d2: 0.8286
+  r2: 0.0332
+  distinct_4: 0.9986
+  repetition_4: 0.0007
+  d4: 0.9986
+  r4: 0.0007
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps2_lam0.8_temp0.94_gpu1.json

eval/h2_mixed_latest_test/logs/pps2_lam0.8_temp0.96_gpu0.log

@@ -0,0 +1,53 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from cli:/home/sunhc/diffusion/sad/configs/block_ar_owt_b32_h2_mixed.yaml
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.8, 0.8]
+leaf_temperature = 0.96
+Generating 128 samples (L=512)...
+  sampled 128/128 (steps this call: 694)
+Average denoising rounds per sample: 694.00
+First sample preview: 'Microsoft CEO, Dennis Hewitt in his keynote speech this week called layoffs "unthinkable and unary" and What he reported'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 5.019142003605953,
+  "median_nll": 4.467703580856323,
+  "ppl": 151.28144915675097,
+  "acc": 0.24060150375939848,
+  "tokens": 65170,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "model_type": "block",
+  "level_lambdas": "0.8,0.8",
+  "avg_steps": 694.0,
+  "positions_per_step": 2,
+  "leaf_temperature": 0.96
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps2_lam0.8_temp0.96_gpu0.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps2_lam0.8_temp0.96_gpu0.json
+Diversity metrics for pps2_lam0.8_temp0.96_gpu0.json:
+  distinct_1: 0.3321
+  repetition_1: 0.3117
+  d1: 0.3321
+  r1: 0.3117
+  distinct_2: 0.8361
+  repetition_2: 0.0299
+  d2: 0.8361
+  r2: 0.0299
+  distinct_4: 0.9982
+  repetition_4: 0.0012
+  d4: 0.9982
+  r4: 0.0012
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps2_lam0.8_temp0.96_gpu0.json

eval/h2_mixed_latest_test/logs/pps2_lam0.8_temp0.96_gpu3.log

@@ -0,0 +1,53 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from cli:/home/sunhc/diffusion/sad/configs/block_ar_owt_b32_h2_mixed.yaml
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.8, 0.8]
+leaf_temperature = 0.96
+Generating 128 samples (L=512)...
+  sampled 128/128 (steps this call: 704)
+Average denoising rounds per sample: 704.00
+First sample preview: " sure, you'll choose.\n\nFavor white. Range can be $necky. Then vary the widest. Exclusively in store TICKS TO Configure t"
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 4.978640177831968,
+  "median_nll": 4.449566602706909,
+  "ppl": 145.27669680313952,
+  "acc": 0.24482377841170516,
+  "tokens": 65202,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "model_type": "block",
+  "level_lambdas": "0.8,0.8",
+  "avg_steps": 704.0,
+  "positions_per_step": 2,
+  "leaf_temperature": 0.96
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps2_lam0.8_temp0.96_gpu3.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps2_lam0.8_temp0.96_gpu3.json
+Diversity metrics for pps2_lam0.8_temp0.96_gpu3.json:
+  distinct_1: 0.3278
+  repetition_1: 0.3168
+  d1: 0.3278
+  r1: 0.3168
+  distinct_2: 0.8327
+  repetition_2: 0.0312
+  d2: 0.8327
+  r2: 0.0312
+  distinct_4: 0.9980
+  repetition_4: 0.0010
+  d4: 0.9980
+  r4: 0.0010
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps2_lam0.8_temp0.96_gpu3.json

eval/h2_mixed_latest_test/logs/pps2_lam0.9_temp0.8_gpu7.log

@@ -0,0 +1,52 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from cli:/home/sunhc/diffusion/sad/configs/block_ar_owt_b32_h2_mixed.yaml
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.9, 0.9]
+leaf_temperature = 0.8
+Generating 128 samples (L=512)...
+  sampled 128/128 (steps this call: 606)
+Average denoising rounds per sample: 606.00
+First sample preview: '\nPolice say the habit of drifting in, from water, streams and streams driven by burning cars, is colloquially known as “'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 32/128
+  scored 64/128
+  scored 96/128
+  scored 128/128
+{
+  "avg_nll": 3.702768828146601,
+  "median_nll": 2.8821914196014404,
+  "ppl": 40.559451180823665,
+  "acc": 0.354950692759055,
+  "tokens": 65102,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "level_lambdas": "0.9,0.9",
+  "avg_steps": 606.0,
+  "positions_per_step": 2,
+  "leaf_temperature": 0.8
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps2_lam0.9_temp0.8_gpu7.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps2_lam0.9_temp0.8_gpu7.json
+Diversity metrics for pps2_lam0.9_temp0.8_gpu7.json:
+  distinct_1: 0.2659
+  repetition_1: 0.3970
+  d1: 0.2659
+  r1: 0.3970
+  distinct_2: 0.7519
+  repetition_2: 0.0752
+  d2: 0.7519
+  r2: 0.0752
+  distinct_4: 0.9895
+  repetition_4: 0.0076
+  d4: 0.9895
+  r4: 0.0076
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps2_lam0.9_temp0.8_gpu7.json

eval/h2_mixed_latest_test/logs/pps2_lam0_temp0.8_gpu2.log

@@ -0,0 +1,51 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [0.0, 0.0]
+leaf_temperature = 0.8
+Generating 128 samples (L=512)...
+  sampled 64/128 (steps this call: 256)
+  sampled 128/128 (steps this call: 256)
+Average denoising rounds per sample: 256.00
+First sample preview: ' hotel bookings lost interest in the second half of the year after another 19-year-old student suffered an overdose in J'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 3.3971109011650245,
+  "median_nll": 2.6199722290039062,
+  "ppl": 29.877655734069887,
+  "acc": 0.37552833407620306,
+  "tokens": 65063,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "level_lambdas": "0,0",
+  "avg_steps": 256.0,
+  "positions_per_step": 2,
+  "leaf_temperature": 0.8
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps2_lam0_temp0.8_gpu2.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps2_lam0_temp0.8_gpu2.json
+Diversity metrics for pps2_lam0_temp0.8_gpu2.json:
+  distinct_1: 0.2232
+  repetition_1: 0.4462
+  d1: 0.2232
+  r1: 0.4462
+  distinct_2: 0.6930
+  repetition_2: 0.1038
+  d2: 0.6930
+  r2: 0.1038
+  distinct_4: 0.9848
+  repetition_4: 0.0122
+  d4: 0.9848
+  r4: 0.0122
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps2_lam0_temp0.8_gpu2.json

eval/h2_mixed_latest_test/logs/pps2_lam1_temp0.8_gpu3.log

@@ -0,0 +1,51 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [1.0, 1.0]
+leaf_temperature = 0.8
+Generating 128 samples (L=512)...
+  sampled 64/128 (steps this call: 633)
+  sampled 128/128 (steps this call: 619)
+Average denoising rounds per sample: 626.00
+First sample preview: ' Novemberó, Pan, Kyana, Picá and Cocoding. You can check out the Loop’s history of graffiti, or view a text list of graf'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 3.7846832664012395,
+  "median_nll": 2.987057089805603,
+  "ppl": 44.02172519093987,
+  "acc": 0.3443800055233361,
+  "tokens": 65178,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "level_lambdas": "1,1",
+  "avg_steps": 626.0,
+  "positions_per_step": 2,
+  "leaf_temperature": 0.8
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps2_lam1_temp0.8_gpu3.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps2_lam1_temp0.8_gpu3.json
+Diversity metrics for pps2_lam1_temp0.8_gpu3.json:
+  distinct_1: 0.2723
+  repetition_1: 0.3864
+  d1: 0.2723
+  r1: 0.3864
+  distinct_2: 0.7657
+  repetition_2: 0.0654
+  d2: 0.7657
+  r2: 0.0654
+  distinct_4: 0.9943
+  repetition_4: 0.0030
+  d4: 0.9943
+  r4: 0.0030
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps2_lam1_temp0.8_gpu3.json

eval/h2_mixed_latest_test/logs/pps2_lam1_temp0.9_gpu6.log

@@ -0,0 +1,51 @@
+`torch_dtype` is deprecated! Use `dtype` instead!
+Using config from checkpoint:/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt
+Loaded SAD checkpoint: /home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt (step=500000)
+level_lambdas = [1.0, 1.0]
+leaf_temperature = 0.9
+Generating 128 samples (L=512)...
+  sampled 64/128 (steps this call: 686)
+  sampled 128/128 (steps this call: 677)
+Average denoising rounds per sample: 681.50
+First sample preview: 'Earlier this year, gender was being enforced with our calendar, but this year that is correct. We know it with most peop'
+Loading eval LM:        /home/sunhc/diffusion/sad/models/gpt2-large
+Loading eval tokenizer: /home/sunhc/diffusion/sad/models/gpt2-large
+
+Eval LM loaded (774,030,080 params)
+Scoring samples under eval LM...
+  scored 64/128
+  scored 128/128
+{
+  "avg_nll": 4.535560568098103,
+  "median_nll": 3.8875941038131714,
+  "ppl": 93.27578808295372,
+  "acc": 0.27707348517404384,
+  "tokens": 65156,
+  "checkpoint": "/home/sunhc/diffusion/sad/outputs/sad_block_ar_b32_h2_mixed/latest.pt",
+  "eval_model": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "eval_tokenizer": "/home/sunhc/diffusion/sad/models/gpt2-large",
+  "num_samples": 128,
+  "generated_seq_len": 512,
+  "mode": "sad_generation",
+  "level_lambdas": "1,1",
+  "avg_steps": 681.5,
+  "positions_per_step": 2,
+  "leaf_temperature": 0.9
+}
+Saved metrics → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/metrics/pps2_lam1_temp0.9_gpu6.json
+Saved samples  → /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/samples/pps2_lam1_temp0.9_gpu6.json
+Diversity metrics for pps2_lam1_temp0.9_gpu6.json:
+  distinct_1: 0.3009
+  repetition_1: 0.3398
+  d1: 0.3009
+  r1: 0.3398
+  distinct_2: 0.8093
+  repetition_2: 0.0405
+  d2: 0.8093
+  r2: 0.0405
+  distinct_4: 0.9974
+  repetition_4: 0.0018
+  d4: 0.9974
+  r4: 0.0018
+  num_samples: 128.0000
+Saved -> /home/sunhc/diffusion/sad/eval/h2_mixed_latest_test/diversity/pps2_lam1_temp0.9_gpu6.json